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Executive Summary 


Over the past decade, the Agency has placed less emphasis on testing and has increasingly relied 
on computational methods to assess durability and damage tolerance (D&DT) behavior when 
evaluating design margins for fracture-critical components. With increased emphasis on 
computational D&DT methods as the standard practice, it is paramount that capabilities of these 
methods are understood, the methods are used within their technical limits, and validation by 
well-designed tests confirms understanding. The D&DT performance of a component is highly 
dependent on parameters in the neighborhood of the damage. This report discusses D&DT 
method vulnerabilities in terms of three important local parameters: 


1. Local length scales (structural and material) within the micromechanics regime. 

2. Local environments that have first-order influences on D&DT behavior. The report only 
discusses mechanical environments (stress and strain). Other local environments (i.e., 
thermal, chemical, radiation, etc.), while not discussed herein, must also be considered when 
deemed critical to D&DT performance. 

3. Local material behavior (i.e., anisotropy, properties, damage mechanisms, etc.), that are often 
not properly or thoroughly considered during the test and analysis design process. 


The lack of understanding of any of the local parameters listed above have led to incomplete and 
erroneous estimates of D&DT design capability. In this report, several vulnerable engineering 
practices related to estimating the D&DT of fracture-critical components are discussed. The 
report also discusses why D&DT method vulnerabilities will rapidly increase with new weight 
saving designs, advanced materials, and unique fabrication processes. The challenge of re- 
tooling D&DT methods is rooted in the need to rigorously address designs having complex 
material length scales and associated local environments that cannot be simulated by current 
D&DT analysis and test methods. 


The objectives of the report are to expose potential vulnerabilities in the current D&DT 
engineering standard practice and to underscore the need for a multi-disciplinary re-tooling effort 
of the Agency’s D&DT engineering predictive practices. The Structures, Materials, and 
Nondestructive Evaluation (NDE) disciplines must collectively work to re-tool the D&DT 
standard engineering practice. Three primary areas need to be emphasized. First, understanding 
and defining the technical limits of current engineering D&DT methods is paramount to ensuring 
their appropriate use and the reliability of their results. This area is associated with technical 
rigor: understanding of the assumptions and limitations of the current D&DT methods and 
establishing effective communication interfaces among key disciplines. Second, it is necessary to 
develop advanced engineering computational tools for D&DT applicable across multiple 
material length scales when current engineering practices are inappropriate. This area is 
associated with developing and maintaining an awareness of potential cross-disciplinary 
influences on a given design and developing computational materials (CM) models to address 
them. Third, it is also important to develop material length-scale-appropriate validation testing 
and inspection methods. This area is associated with identifying and providing the appropriate 
material testing methods, property data, and inspection technologies to validate new analytical 
methods and understand local material behavior. Recommendations suggest re-tooling the 
Agency’s D&DT standard engineering practice. 
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1.0 Introduction 


NASA’s human space mission goals have focused on Low-Earth Orbit (LEO) for 60 years 
spanning the Mercury Program through International Space Station (ISS). This LEO paradigm 
for space system reliability is based on the following assumptions: 1) Limited lifetime (ISS 
designed for 15 year life and extended to nearly 30 years), 2) Replacement (components and 
parts are re-supplied as needed), 3) Unknown-unknowns (rapid fixes are transported from earth 
as needed), 4) Safe haven on Earth (one day return to Earth). Once we leave LEO and travel 
millions of miles, a new deep-space paradigm will need to be constructed based on different 
assumptions: 1) Extended lifetime (all assets are valuable in deep space), 2) Limited replacement 
(limited spare components and parts), 3) Unknown-unknowns (robust detection, analysis and 
repair), 4) Returning to Earth as a safe haven is a highly complex and dangerous 

endeavor. When comparing the LEO and deep space paradigm assumptions, it is obvious that 
ultra-reliable technologies will be required for deep space travel. These ultra-reliable 
technologies can be developed and deployed only through the understanding that comes from 
next generation D&DT methodologies. 


The D&DT certification pendulum has swung too far as the Agency is relying too much on 
existing computational tools and standard engineering practices when evaluating design margins 
of some unique fracture-critical components. With increased emphasis on computational D&DT 
tools as standard engineering practice, it is paramount that these methods and tools be used 
within their technical limitations. Currently, complex local environments and local material 
behavior are often not properly or thoroughly considered, thereby leading to incomplete and 
erroneous estimates of the local D&DT response. The problem often stems from the fact that 
most analytical and computational procedures are currently formulated using conventional 
continuum mechanics theories that cannot treat failure mechanisms and modes arising from local 
material properties and local structural configuration, respectively. New designs, materials, and 
fabrication technologies push the limits of our modeling and understanding to demonstrate 
engineering predictive practices for D&DT. The reasons stem from the fact our models are not 
good enough, our material characterization is not sufficient, our understanding of damage and 
failure mechanisms is inadequate, and our testing is not extensive enough to understand the 
damaging effects of local environments. However, given these limitations and lack of true 
understanding of the material response, failures are still rare events. This is likely due to using 
conservative factor of safety, using well-characterized materials having statistically relevant 
material data, and perhaps ultra-conservatism related to loads and environments. We postulate 
that the Agency is entering a new era of deep-space travel that will require new weight-saving 
designs that must survive less tractable environments. 


New designs come with unforeseen risks. Managing these unforeseen risks is becoming a real 
concern when the Agency’s current technical approach is insufficient in some critical D&DT 
areas. The current engineering practice tries to minimize unforeseen risks with computational 
methods and tools that may be pushed to or even beyond their applicable limits and checked with 
a level of testing that may be insufficient. As a consequence, the limits of proven engineering 
practices are exceeded when applied to advanced designs that use new materials or fabrication 
processes. Sometimes these tools are applied without a full understanding of their technical 
limits or without appropriate testing and data to characterize the material response for local 
environments and loads. Understanding these limits results in properly using the current D&DT 
engineering predictive tools and recognizing the need for new D&DT engineering methods; that 
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is, re-tooling. The need for analysis and testing methods accounting for a range of material 
length scales is generally not recognized by the engineering community because of the explosive 
increase in computing resources and ease-of-use interfaces to commercial finite element analysis 
tools. Modern engineering tools can easily generate a refined finite element model (FEM) 
wherein the finite element size is reduced to very small scales but generally ignores 
microstructural features. These features can become important at such small scales and can 
influence the D&DT assessment. 


The diagram shown in Figure 1.1 illustrates the important relationships of length scale with 
D&DT engineering practices and CM based methods. At the larger structural (macro) length 
scale (large arrow pointing upward to the right in the figure), structural mechanics, continuum 
mechanics, and linear elastic fracture mechanics (LEFM) accurately estimate global structural 
performance within their range of applicability. In this macro-regime, global and local finite 
element stress analysis models are developed and the results are used to assess D&DT capability 
at the macro-scale level with little regard to material length scales (see Appendix A). This 
macro-regime approach forms the basis for most of the Agency’s current D&DT standard 
engineering practice and a large portion of the aerospace D&DT community. The methods are 
well understood, the material response is usually well characterized, the fabrication processes are 
well documented, and the environments are accurately defined for the macro-scale response. 
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Figure 1.1. D&DT analysis and test methods as a function of length scale. 


As critical structural length scales decrease and approach microstructure dimensions (micro 
length scale), the global continuum mechanics approach applicable at the macro-scale no longer 
applies. Multi-scale approaches are being introduced wherein the localization of micro-structural 
details is evaluated using representative unit volumes coupled with assumptions related to 
repeatability or periodicity. The results are then communicated to the macro-level through a 
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homogenization procedure. The influences of micro-structural effects are then represented as 
effective or apparent material properties using current standard engineering practices as indicated 
by the dark lines at the base of the upward pointing arrow in the Figure 1.1. Such multi-scale 
approaches tend to leverage the macro-scale D&DT approaches but potentially risk missing 
microstructural influences (e.g., structural length scale approaching the grain-boundary length 
scale). These approaches provide added insight into the global response at the expense of 
understanding damage initiation, propagation, and material failure at the microstructural level. 


Micromechanics-based methods, local stress and strain state determination, local damage 
initiation and propagation understanding, and local microstructure-based material properties 
must be considered to assess D&DT performance for emerging new designs, new materials, and 
new fabrication processes. The arrow pointing downward to the left in Figure 1.1 is an 
indication that some Agency research activities in CM methods are underway but lack sufficient 
impetus and resources to overcome decades of neglect by the Agency. Fundamental research to 
provide engineering D&DT methods and tools for the emerging designs, materials, and 
fabrication processes has not been conducted at NASA. Conversely, other government agencies 
such as the Department of Energy (DOE) and the Department of Defense (DOD) have provided 
substantial and sustained funding for several decades to develop new mechanics models and 
computational approaches. These efforts are aimed at understanding the interplay between 
different material length scales and between structural and material length scales. However, there 
are still numerous shortcomings with the research methods and the transition from a research 
solution to an engineering capability represents a near empty set — a critical technology gap 
influencing near-term D&DT re-tooling. In recent years, it has become apparent that the 
Agency’s current D&DT macro-based engineering practice has been inappropriately applied to 
estimate the performance of flight components that contain a critically small structural length 
scale and material (microstructural) length scale. This scale is depicted in the micromechanics 
regime highlighted by the center cloud shown in Figure 1.1 — the regime where the current 
D&DT analysis and test methods fall short. 


Examples illustrated in Figure 1.2 are used to underscore the Agency’s engineering practice 
shortfall. New complex designs have incorporated small/thin structural elements (1.e., metallic 
Composite Overwrapped Pressure Vessel (COPV) liners), complex additive-manufactured 
components, and complex flexible structures (e.g., inflatable structures and habitats) whose 
D&DT performance cannot be reliably estimated by current engineering practices. In addition, 
new complex materials (i.e., gradient microstructure (multifunctional), advanced composites 
(polymer, metallic, ceramics), structural applications of non-structural materials (thermal 
protection systems, foams), etc.) containing complex microstructures will require new 
computational-materials-based methods to address even smaller/shorter length scales and 
properly estimate D&DT behavior. Because the Agency is currently focusing on new complex 
weight-saving structural concepts that require micromechanics length scale methods, the Agency 
must critically assess its D&DT analysis and test capabilities to insure that the proper D&DT 
understanding is developed prior to flight. 
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Figure 1.2. Examples of D&DT methods short-fall within the micromechanics length scale 
regime. 
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1.1 Objectives 


The first objective of this report is to identify potentially vulnerable engineering practices related 
to estimating and predicting the response of fracture-critical components that challenge the 
current standard D&DT methods. These challenges result from the need to address multiple 
material length scales in assessing D&DT requirements for future aerospace systems. 
Vulnerabilities are associated with using the current D&DT analysis and testing methods that are 
primarily focused on macro-scale responses to provide accurate and valid estimates of material 
behavior across different material length scales and to provide a greater understanding of the 
influence of local environment(s). This report discusses some of the causes for the D&DT 
analysis and testing technology gap. 


The second objective of this report is to highlight the need for heightened awareness by the 
engineering community of the limitations of the current D&DT tools as the Agency pursues new 
materials and processes required for complex aerospace designs.” While pockets of CM research 
and development are underway within the Agency, current D&DT engineering practice relies on 
commercially available existing tools and procedures with those results deemed sufficient 
rationale to reduce a factor of safety. A critical technology gap exists between these CM 
research and development efforts and their integration or transition to D&DT engineering tools. 
It is hoped that this report will drive home the importance of and reinvigorate the establishment 
of a long-term, sustained multi-disciplinary effort to bolster the Agency’s D&DT research and 


> The NASA Engineering and Research communities have been aware of some of these limitations. A Foundational 
Engineering Science initiative within the Agency was attempted a number of years ago but has lost focus. 


NESC Document #: NESC-NPP-17-01 6 


tools development required by the 21“ century aerospace community — a re-tooling of the 
Agency’s D&DT engineering predictive practices. 


1.2 Path Forward 


The Structures, Materials, and NDE disciplines must collectively work to re-tool the D&DT 
standard engineering practice. Understanding our current tools and their limits is a critical first 
step in the re-tooling process — our existing tools are valid for many cases. Communication 
across and between disciplines needs to be strengthened and enhanced, in particular, a common 
vocabulary and language must evolve across workforce and management levels. Examples of 
vocabulary and language disconnects include the perception that a finite element meshing 
equates to modeling or that finite element meshing based on solid geometry definition defines 
high-fidelity. Collaboration between disciples is needed as D&DT is such a broad sub-discipline 
that no single discipline has the answer. New D&DT tools (tests and methods) applicable to new 
designs, materials, and processing need to be developed and transitioned from D&DT research 
topics to new engineering D&DT tools. Finally, education is needed to enlighten Agency 
Programs of the importance of using the appropriate D&DT methodologies and that the D&DT 
community understands the capabilities and limitations of the D&DT tools. 


Furthermore, as new designs, materials, and fabrication processes emerge, the importance of 
material length scales in the micro- and nano-range is anticipated to increase. Understanding the 
influence of these effects on the D&DT of future flight systems will necessitate further research 
in CM for discrete continuum mechanics, molecular dynamics and atomistic mechanics. It is 
anticipated that such long-term efforts will be enabled by parallel developments in computing 
systems — both hardware and software. These future advances must be developed with the 
perspective of providing future generations of D&DT methods and tools — re-tooling. 


Three primary areas are recommended as needing attention. First, understanding and defining 
the technical limits of current engineering D&DT methods is paramount to ensuring their 
appropriate use and the reliability of their results. This area is associated with technical rigor: 
understanding the assumptions and limitations of the current D&DT methods and establishing 
effective communication interfaces among key disciplines. Second, it is necessary to develop 
advanced engineering computational tools for D&DT applicable across multiple material length 
scales when current engineering practices are inappropriate. This area is associated with 
developing and maintaining an awareness of potential cross-disciplinary influences on a given 
design and developing CM models to address them. Third, it is also important to develop 
material length-scale-appropriate validation testing and inspection methods. This area is 
associated with identifying and providing the appropriate material testing methods, property 
data, and inspection technologies to validate new analytical methods and understand local 
material behavior. These recommendations suggest re-tooling the Agency’s D&DT standard 
engineering practice. 
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2.0 Background 


Over the past decade, the Agency has placed less emphasis on testing and has increasingly relied 
on analytical and computational procedures to understand and assess durability? and damage 
tolerance* (D&DT) behavior when evaluating design margins for fracture-critical components. 
While this trend has the perceived advantage of reducing schedules and costs, the D&DT 
community needs to recognize and address potential material length-scale (see Appendix A) 
issues as they relate to D&DT methods to not compromise safety unknowingly. 


Currently, complex local environments (highly localized stress and strain responses) and material 
behavior are often not considered, thereby leading to incomplete and erroneous estimates of the 
local D&DT response. The problem often stems from the fact that most analytical and 
computational procedures are currently formulated using conventional continuum mechanics 
theories> that cannot treat failure mechanisms arising from local material microstructural effects 
(see Ref. 1). Consequently, results extracted from such continuum-based global analysis models 
can lead to erroneous D&DT results when applied to designs and materials exhibiting length- 
scale dependencies. One example of length-scale dependencies arises either from small defect 
sizes or small uncracked ligaments both of which can violate LEFM assumptions®. Another 
example arises from complex material microstructures that cause anisotropic material behavior 
leading to stress concentrations along grain boundaries, and interactions between failure 
mechanisms (see Ref. 1). 


Often the current engineering practice advocates developing detailed, large-scale finite element 
analysis (FEA) models of a structural component under the premise that material length scales 
are related to the size of a finite element within the mesh. Such global FEA models impose 
boundary conditions, loads, and environments defined at the macro-level to estimate strains and 
then determine stresses using a continuum-based material model tied to standard material 
properties derived from coupon tests. These trends are described in Section 3.0. Some FEA 
models may include a defect or crack wherein local deformation, stress, and strain estimates are 
extracted. These estimates are used as input to fracture mechanics and fatigue tools to assess 
D&DT capability. Results from such FEA models and LEFM estimates are frequently perceived 
to be accurate and credible even though they could be beyond the capability of the FEA models. 
The results were likely computed using material models developed and validated for macro-scale 
continuum problems where scale independence was demonstrated to apply. Examples from 


3 The ability of the structure to resist cracking, corrosion, thermal degradation, delamination, wear, and the effects of foreign 
object damage for a prescribed period of time (from MPCV 70135, Orion Multi-Purpose Crew Vehicle (MPCV) Program: 
Structural Design and Verification Requirements (SDVR), Rev. A, June 9, 2015). 

4 The attribute of a structure that permits it to retain its required residual strength for a period of unrepaired usage after the 
structure has sustained specific levels of fatigue, corrosion, accidental, and/or discrete source damage (from MPCV-70135). 

> Conventional (scale independent) continuum mechanics theories assume a length scale generally related to the structure being 
analyzed and exhibit an independence from material length scales. Scale-dependent continuum mechanics models take into 
account the dependence of a material's response on the physical phenomena occurring at the appropriate material length scales. 

© The underlying fundamental assumptions of LEFM are (i) the material away from the crack tip undergoes elastic deformations; 
(ii) the plastic region ahead of the crack tip is small in relation to the length of the crack; (iii) deformations outside the small 
plastic regions are presumed to remain elastic; and (iv) the material is presumed to have a grain structure such that the integrated 
effects of the grains in these regions act as a continuum. 
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previous NASA Engineering and Safety Center (NESC) assessments are cited in Section 4.3 of 
this report. These examples show when length-scale-based tenets have been unrecognized or 
ignored and hence violated by a global analysis approach advocated by the current D&DT 
standard engineering practice. 


Such engineering practices related to estimating/predicting the safe life of fracture critical 
components’ whose design and fabrication challenge standard D&DT methods raise potential 
vulnerabilities. Vulnerabilities are associated with current D&DT analysis and testing methods 
focused on macro-scale responses requiring accurate estimates of material behavior across 
different material length scales and greater understanding of local environment(s). These 
vulnerabilities are becoming more evident with the development and integration of advanced 
material systems and material architectures (e.g., advanced alloys, multi-directional composites, 
multifunctional materials), with the application of advanced fabrication and manufacturing 
technology (e.g., advanced fiber placement and additive manufacturing), and with the increased 
demand for reduced weight and increased performance (to support interplanetary missions and 
human outposts). 


3.0 Current Practice 


Current engineering standard practice for D&DT predictions is unable to provide, through 
analysis and testing, the understanding required to ensure safe and reliable flight of new complex 
vehicle designs based on advanced materials and fabrication processes. Once this inadequacy is 
recognized, the D&DT predictions fail to capture the required technical understanding of the 
small length-scale failure mechanism due to test and inspection limitations. This problem is 
exacerbated either by the enormity of the computational task associated with microstructural 
modeling and simulation or by the complexity of defining and obtaining the relevant material 
properties of advanced materials and fabrication processes. In fact, quantifying material 
properties in complex materials requires developing new testing and inspection methods and 
advanced CM models. These models must incorporate the fundamental microstructural interface 
properties across multiple material scales that often control the operative failure mechanisms. 
Thus, it is concluded that a new class of damage-science-based predictive tools is required to 
accurately represent the damage initiation and failure mechanisms necessary for accurate D&DT 
predictions and to ensure mission success and safety. In the interim, the role of testing for 
characterization and certification needs to be elevated. Moreover, maturing such advanced 
D&DT methods from a research and development framework to standard engineering practice 
(i.e., a new normal) is crucial for future Agency spaceflight missions that exploit the emerging 
materials and fabrication technologies required to meet ever higher performance requirements 
safely. 


The Agency’s Safety and Mission Assurance (S&MA) organization has succinctly captured the 
proper tenets to ensure understanding of risk. These tenets are based on technical rigor rooted in 
physics-based approaches to both testing and analysis and they outline the fundamentals of good 
engineering practice. The following seven tenets from S&MA have been slightly modified to 
suit the notions developed herein and are listed here with multiple material length scales included 
as one key factor. 


7 Fracture critical components are parts whose individual failure, caused by the presence of a crack, is a catastrophic hazard and 
that requires safe-life analysis or other fracture control assessment to be shown acceptable for flight. Ref: NASA-STD-S019A. 
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Solid technical understanding — Physics-based or root-cause understanding of technical issues 
associated with the design, based on sound engineering anchored with testing under relevant 
loading and environmental conditions. This tenet requires recognizing the limitations of 
standard practice tools and of the potential impact that designs with multiple material length 
scales may have on the tools’ applicability. 


Condition relative to experience base — Experience is based on tests and validated analytical 
tools and is influenced by the background and seasoning of test engineers and analysts. This 
tenet requires an awareness and understanding of advances in material science and fabrication 
technology by inclusion of next generation D&DT methods and tools. 


Bounding case(s) established — Using physics-based understanding from validated analytical 
tools and anchored with tests, determine bounding case(s) for the mission based on a thorough 
understanding of environments, mission requirements, and the use of non-deterministic® 
approaches. This tenet requires uncertainty quantification to understand risk and should include 
an assessment of appropriate material length scales. 


Self-limiting aspect — Understand the physical reasons why a bounding case is truly bounding or 
show the part is fail-safe — that is, damage tolerant’. This tenet requires the material response to 
be consistent across different material length scales and accounts for local environments. 


Margins understood — Maintain adequate margins of safety, ideally, not substantially reduced 
from the baseline. Margins are essential to protect the system or vehicle from potential threats 
that may not be well understood. This tenet requires an understanding of the dependence (or 
independence) of the margin of safety assessments across material length scales with potentially 
different failure mechanics emanating from different scales. 


Risk Assessment based on test data and analysis -Minimize engineering judgment. As the 
complexity of the new designs increases, and as new materials and manufacturing processes are 


incorporated, engineering judgment may be inadequate or even detrimental. This tenet requires a 
re-tooling of standard engineering practices to accommodate emerging material systems and 
fabrication technologies and address multiple material length scales. 


Consider interactions between critical elements/conditions —Understand and mitigate design 
sensitivities. The design should be as robust as possible without known potential susceptibilities 
that could compromise safety. This tenet requires thorough assessment of the known unknowns 
and recognition that unknown unknowns remain for high-performance designs using advanced 
materials, concepts, and technologies. 


Because these tenets are anchored by rigorous engineering practices, the engineering community 
must constantly ensure their computational tools and their inputs are truly appropriate so that 
system performance and risk can be correctly estimated with confidence and managed 
appropriately. While not a new problem, it has been exacerbated by the integration of new 
materials, complex designs, and new fabrication processes. Current engineering computational 
tools and tests methods are not capable of quantifying design margin and associated risk for 


8 These non-deterministic approaches would deviate from current deterministic continuum mechanics methods to accommodate 
effects such as different material length scales when predicting material and structural responses. 

° A fail-safe assessment demonstrates structural redundancy in the event of loss of one load path and includes evaluating 
structural margins for redistributed static and dynamic loadings. 
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these new cases when multiple material length scales are important. Sometimes, the need to 
model multiple length scales is not recognized. This limitation and challenge is particularly true 
when test data are limited, when environments cannot be simulated in ground tests, or when 
operational environments are unknown or uncertain. Often, statistically relevant material 
allowables cannot be generated due to cost, schedule, or the lack of available material to test. In 
addition, newer materials may necessitate the development of alternate or additional material 
tests or characterization methods to determine actual material allowables applicable for that 
material system or manufacturing approach (e.g., additive manufacturing, advanced fiber 
placement). These result in different material length scales to be evaluated for D&DT. 


Current engineering practices often use large-scale global finite element analyses of a test 
coupon, structural element, or component (i.e., macro-scale modeling) to compute local 
response. The premise is that local stress states may be obtained using local strain estimates and 
continuum-based material models and global response metrics. Advances in computation power 
have enabled global finite element analyses to be performed with finer and finer meshes with the 
assumption that the more refined the mesh, the more accurate the local stress prediction will be. 
However, bigger is not always better and does not guarantee a valid solution. Finite element 
mesh refinement is needed to demonstrate convergence of the discrete analysis model, its 
underlying assumptions, and response parameters as part of the verification process. Some 
engineers may even assume that accurate microstructural response can be predicted as the mesh 
size (characteristic length of a finite element) approaches the microstructural scale of the 
material. These assumptions are not correct and lack the required technical understanding. The 
computations become increasingly compromised without a basic understanding of length-scale- 
related material properties and strain-localization/microstructure that govern damage processes 
combined with the false assumption that a more refined FEM leads to more understanding. The 
material length scales (see Appendix A) referred to herein are associated with critical local 
properties and mechanisms that must be incorporated into the CM model within the analysis 
tools, and are not associated with the characteristic length of a finite element (mesh size) within 
the analysis model. 


The need for more efficient spacecraft designs for long-duration robotic and human space flight 
is challenging the Agency’s D&DT standard engineering practices. Central to safe long-duration 
space flight is understanding D&DT and providing a credible D&DT predictive capability. 
Compounding the never-ending challenges of space flight, such as the one-of-kind vehicle 
designs that must survive unique extreme environments, is the current challenge of long-duration 
human exploration. The engineering community must be reminded of the limits of current 
engineering analysis tools, testing methods, and associated standard practices and must be 
vigilant in their use. Specifically, the computational mechanics community recognized the 
significance of the previously mentioned improper or inappropriate use of the current standard 
engineering methods, and took action to understand and resolve these issues. The Agency 
should proceed with identifying and developing advanced computational engineering tools 
validated by appropriate tests and NDE inspection methods, thereby re-tooling the Agency’s 
D&DT capabilities and enhancing engineering predictive practices for future aerospace systems. 
This will create a new normal of standard engineering practice. 
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4.0 Motivation and Challenges 


The Agency’s Human Space Flight Programs are moving forward with qualifying commercial 
partners’ LEO vehicles, new launch/space vehicle designs, and habitat concepts for deep-space 
exploration. The Agency’s Robotic Program is also concentrating on more efficient, but even 
more complex designs subject to extreme environments and interplanetary missions. As these 
complex Agency missions mature, it is imperative that engineering methods and simulation 
results that mimic system response to environments and local D&DT phenomena, are scrutinized 
to ensure that thorough technical understanding is established prior to the design, development 
and certification phases. Over the past decade, new designs have become understandably more 
complex because of the need for increased efficiency and performance (e.g., reduced weight, 
enhanced load-carrying capability, improved thermal performance, etc.). Each new design 
typically contains new technical complexities and associated risks that engineering must not only 
quantify, but more importantly recognize and understand. Programs then routinely accept risk 
with the expectation that engineering has addressed all knowledge gaps in understanding the 
D&DT aspects of the new designs and fabrication processes, in characterizing material behavior 
and failure mechanisms, and in identifying global and local environments and operational 
requirements. They expect that the appropriate and applicable D&DT computational tools and 
test methods were used to properly assess design safety margins. However, on occasions the 
current engineering process has failed to recognize one or more knowledge gaps, or has failed to 
be astute enough to ask questions that may challenge the status quo. Often times, engineering 
judgment derived from past experiences does not contain sufficient technical understanding or 
rigor when applied to these new designs or new configurations or new materials with the system 
having high-performance requirements for mission success. 


4.1 Traditional Analysis Paradigm versus Required Technical Understanding 


New and more complex designs are subject to high-performance operational requirements and 
are potentially operated in extreme environments with minimal or no possibility for inspection or 
repair once deployed. These new designs use unique light-weight configurations, exploit 
advanced materials and innovative fabrication processes, and challenge our ability to provide a 
credible and accurate design safety margin assessment. This issue has surfaced at an increasing 
rate suggesting that current D&DT engineering computational methods used by the Agency, and 
its contractors, are becoming outmoded for some advanced fracture-critical designs. Lack of 
recognition that some of our conventional computational tools are inappropriate can have serious 
consequences on mission success as predictive analyses are being increasingly relied upon with 
less emphasis on costly testing. Using outmoded and inappropriate computational tools for new 
complex designs and materials directly impacts the Agency’s ability to predict safe-life 
performance accurately. New complex design configurations that have been enabled by 
advanced materials and innovative processes (e.g., additive manufacturing, component designs 
with small length scale, micro-electronics, etc.) have created a gap in understanding local 
material response and its impact on material modeling and testing. Current computational 
methods are not capable of developing the fundamental understanding at the critically small 
length scales needed to predict appropriate material response metrics (Ref. 1-8). 
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The lack of understanding is exacerbated by the persistent using traditional engineering analysis 
paradigms that may not apply where small length scale understanding is required. For example: 


e Inappropriately accepting finite element results based on mesh density, using three- 
dimensional solid geometries developed with computer aided design tools, and using a 
particular FEA tool. The perception that when the smallest element size in a finite element 
mesh approaches the material grain size then microstructural effects are automatically 
included in the model is false. 

e Lack of understanding of fundamental assumptions, boundary conditions, and response 
metrics produced by the computational models. 

e Inability to define and recognize the boundaries of applicability for conventional continuum 
mechanics assumptions. 

e Inappropriate use of LEFM due the violation of its fundamental assumptions. 

e Lack of understanding of fracture mechanics similitude concepts!” leading to the potentially 
inappropriate use of engineering tools such as NASGRO® (Ref. 9). 

e Inappropriate use of finite element stress results (such as a calculated point stress or an 
augmented stress metric such as the von Mises stress!') leading to the inability to understand 
and evaluate local material behavior and ignores local triaxiality and directional dependence. 

e Lack of recognition of the importance of local environments (e.g., local loads, load 
redistribution, and interfacial strain, etc.). 

e Lack of understanding of material length scales and their influence on material behavior at 
multiple length scales such as increased dependence on local material anisotropy and 
nonlinearities at small length scales superimposed on directional dependencies at larger 
length scales. 

e Lack of understanding across disciplines of existing standard practice guidelines such as 
American Society for Testing and Materials (ASTM) E647, Standard Test Method for 
Measurement of Fatigue Crack Growth Rates”. 


10 The Similitude concept implies that, for two different sized cracks subjected to equal stress intensity values (under small-scale 
yielding) in a given material-microstructure-environment system, crack tip plastic zones will be equal in size and the stress and 
strain distributions along the borders of these zones (ahead of the crack) will be identical. Accordingly, equal amounts of crack 
extension (Aa) will be expected. See Ritchie and Suresh (Ref. 3). 


Ito apply the uniaxial stress-strain model, an effective stress is defined typically using the von Mises stress, O',, , which can 


be written in terms of the three principal stresses as: 


1 7 5 aye 
Og =O =—|(c, -0,) +(o0,—-0;) +(6;—-90,) | 


V2 


The von Mises stress criterion compares this effective stress to the yield stress from a uniaxial tension test data and assumes a 
linear elastic, isotropic, homogeneous ductile material and ignores any directional dependencies. 

'2 ASTM E647 (Ref. 10), Paragraph 5.1.5 - The growth rate of small fatigue cracks can differ noticeably from that of long cracks 
at given AK values. Use of long crack data to analyze small crack growth often results in non-conservative life estimates. The 
small crack effect may be accentuated by environmental factors. Cracks are defined as being small when 1) their length is small 
compared to relevant microstructural dimension (a continuum mechanics limitation), 2) their length is small compared to the 
scale of local plasticity (a linear elastic fracture mechanics limitation), and 3) they are merely physically small (<1 mm). Near- 
threshold data established according to this method should be considered as representing the materials’ steady-state fatigue crack 
growth rate response emanating from a long crack, one that is of sufficient length such that transition from the initiation to 
propagation stage of fatigue is complete. Steady-state near threshold data, when applied to service loading histories, may result in 
non-conservative lifetime estimates, particularly for small cracks. 
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These concerns and issues limit the Agency’s ability to develop technical understanding and 
consequently burden engineering’s ability to ensure that risks associated with a new design are 
identified, understood, and properly quantified and that their consequences are communicated 
and managed. 


4.2. Examples: Inability to Assess D&DT 


Based on a review of past Materials and Structures Discipline-related NESC assessments 
conducted over the past 10 years, there is a troublesome trend showing that important 
Engineering methods have been misapplied and/or their results misinterpreted or oversold. 

Often analysis results obtained from refined FEMs are presented with a tacit assumption that the 
results from such an analysis must be accurate since they were executed with one of our 
traditional tools using a very refined FEM — some may say a ‘high-fidelity model’. The 
importance of modeling and simulation standards (such as NASA STD-7009) requiring 
verification and validation (V&V) tends to be down played because of our familiarity with and 
disposition to accept such computed results with limited scrutiny. Shortcuts or even side steps to 
V&V procedures are often taken due to schedule, cost, or staffing. However, as performance 
requirements increase and innovative structural concepts, materials, and fabrication processes are 
incorporated in the design, such confidence in current engineering analysis tools is eroding as the 
structural global response tends to be nonlinear and the local material response tends to involve 
multiple length scales. Specifically, our computational tools must now incorporate appropriate 
materials models that properly account for spatial dependencies (i.e., different material length 
scales) to develop the necessary understanding required to quantify design margin and flight 
risks and an awareness of when different material length scales are applicable. To understand 
the risk of failure, our computational tools must now incorporate capabilities to address a basic 
understanding of material performance (i.e., new material modeling tools to mimic observed 
local material behavior) and spatial dependence (i.e., inherent response dependencies across 
multiple material length scales). These two aspects are described further in the following 
sections. 


4.2.1 Material Performance 


FEMS are used to estimate the global structural system response and to extract the local 
deformation and stress states for use with a fracture mechanics tool, such as NASGRO® (Ref. 9). 
These computational models require using appropriate materials models that replicate global 
structural behavior associated with structural stiffness and deformations as well as attempting to 
mimic local material behavior associated with nonlinear local response, damage initiation, and 
damage propagation. In some cases, engineering practice has dictated developing a local model 
of a critical “hot spot” region using the same length-scale-independent continuum modeling 
approach to better define the local stress state. However, these material models ignore the 
physical basis of the material length scale dependencies. 


Many finite element tools today use standard materials models based on conventional continuum 
theory and provide a single scalar stress metric, such as the effective stress using the von Mises 
assumption applied to ductile and isotropic materials behavior, to assess the local stress state. 
Local stress states are commonly anisotropic, rarely linear, and almost never characterized by a 
scalar quantity. A single scalar stress metric such as the von Mises stress, while convenient and 
common, is an inadequate metric for the local material behavior and its use can contribute to 
misinterpreting the true D&DT capability. Some finite element tools have been customized with 
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limited success using user-defined material models to address microstructural details within a 
CM framework. Unfortunately, applying these new material models requires significant 
validation effort for them to become new standard practice engineering tools, and when applied 
to an actual design, the resulting analysis model becomes computationally intractable using the 
current computational environment. 


4.2.2 Spatial Dependence 


Recognizing and accounting for spatial dependencies associated with the material as dictated by 
the design, by the fabrication or manufacturing process, or by microstructural tailoring for 
performance must be reflected in the materials model integrated within the modeling and 
analysis strategy whether it is a finite element approach or a new analysis paradigm. These 
material models provide a new class of computational tools that ensures accurate estimates of the 
local stresses for the appropriate material length scales (see Refs. 4-8). Capturing these local 
material spatial dependencies analytically becomes more and more critical with the need to 
estimate the local stress state at a different (smaller) material length scale than the standard 
practice FEM has the capacity to simulate. Furthermore, test data to validate such new material 
models are nearly non-existent. Validating the analysis modeling tools by test is needed to 
demonstrate analysis credibility and predictive capability. Such a process leads to identifying 
and understanding contributions to engineering risk thereby leading to more accurate uncertainty 
quantification. 


If the specific design problem requires predicting local stresses, local spatial dependencies 
associated with the material length scales depicted in Appendix A must be considered. 
Examples of such local spatial dependencies are plasticity, anisotropy, multi-directional 
composite, local microstructure details, and other length-scale dependencies (see Ref. 1). As the 
region of interest becomes small, conventional continuum mechanics assumptions coupled with 
material properties obtained from specimen and coupon testing!’ may not be valid for local stress 
prediction due to voids, interfaces, and discrete changes in microstructure. More complex, non- 
traditional materials models are needed to simulate local material behavior needed to quantify 
the local state of stress. Attempts aimed at this technology gap are evident in Refs. 11 through 
21. This understanding is critical to accurately assessing D&DT capability and design margins. 
However, such a material model of a local state may not be available or may not be 
computationally tractable using current engineering tools. As a result, engineering assumptions 
are made to use averaged or smeared ‘apparent’ material properties for both global and local 
response determination. These assumptions are infrequently challenged or revisited, but instead 
are often accepted as an accurate representation of the local stress state based on the local mesh 
refinement (i.e., the finite element size approaches the grain size). 


As the spatial modeling features of the analysis model approach the characteristic dimensions of 
the material’s microstructure, very complex local non-continuum material effects (e.g., discrete 
material anisotropy, discontinuities, grain boundaries and grain orientations) that influence local 
material properties must be considered. This consideration is to ensure that D&DT predictions 
are accurate rather than relying on models using empirically fit parameters. More often than not, 


'3 Such properties would represent micro-structurally averaged or apparent homogeneous mechanical properties for the test 
coupon. These tests are used to characterize macro-scale material properties including directional dependencies associated with 
large-scale structural anisotropy, but are inadequate to characterize the ever-present small-scale material anisotropy (i.e., grain 
boundaries, dislocations). 
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material data for new material models at these length scales are not available. For example, bulk 
tensile properties measured by standard uniaxial tensile testing assume continuum behavior. 
Uniaxial tension testing provides the basic material input to most elastic-plastic material models 
available in today’s engineering analysis tools. However, these uniaxial tensile test results may 
not be applicable to problems involving a multi-axial stress state — even though a refined finite 
element mesh can be developed and a von Mises stress calculated using point stress components. 
For example, local microstructure properties that govern plasticity-driven failure processes'* may 
need to be considered to assess the D&DT capability accurately. Unfortunately, such analysis 
results tend to be interpreted as an accurate representation of the local stress state, when in fact it 
is simply an extraction of a local stress based on global material assumptions (i.e., homogeneity) 
that cloud the D&DT assessment. 


4.3 Examples Illustrating Standard Engineering Practice D&DT Challenges 


An increasing number of NESC D&DT assessments illustrate the importance of recognizing the 
improper use of conventional standard-practice engineering tools and the tendency for blind 
acceptance of their calculated results perhaps due to schedule, cost, lack of understanding, or a 
combination of these factors. The following examples are given to illustrate the reality of the 
situation. Details of these examples are in the cited documents, only findings and 
recommendations from these assessments are cited here to illustrate the issue and its recurring 
nature. 


4.3.1 Example # 1 - D&DT of Composite Overwrapped Pressure Vessel (COPV) Metallic 
Liners 


Human Exploration Programs and Science Programs are both currently using light'* COPV’s 
with 0.030- to 0.040-inch thick metallic liners. The ISS COPV inventory listing is shown in 
Appendix B with two having a liner thickness less than 0.046 inches. The Commercial Crew 
Program (CCP), the Orion Program, and Space Launch Systems Program are considering light 
COPV designs. In addition, robotic science mission programs are considering using light COPV 
metallic liners as thin as 0.010 inch on future robotic missions for the potential weight savings. 
This is understandable, but the presumed minimal weight savings is questionable given that the 
increased risk associated with using a thin-liner COPV design is unknown and unquantified. 
COPVs exhibit two primary risks: one associated with the metallic liner (discussed here as 
Example 1) and the other associated with the composite overwrap (discussed later as Example 
5). 


The risk associated with the thin metallic liners and the ability to predict the D&DT performance 
of the COPV liner is documented in Appendix C and its attachments. This appendix presents a 
series of technical questions that summarizes current unknowns relative to the use of light COPV 
designs for CCP systems and contributed to the release of NESC Technical Bulletin No. 16-02 


'4 Plasticity-driven failure processes — as local component dimensions become small relative to defect size or microstructure 
attributes and local D&DT predictions are necessary, local material properties (non-continuum effects) must be considered. For 
example, anisotropy effects may indicate that brittle grain boundary fracture effects must be considered or the effect of welding 
has influenced local microstructure properties or stress state (weld induced residual stress). Also, as defects approach a surface 
boundary, the stress intensity factor field is affected so that the similitude assumption cited previously in Section 4.1 becomes 
non-applicable. Eventually, localized micromechanical damage processes may dominate the failure condition. 

'S The term “ight” COPV is used herein for COPVs containing thin liners of thickness less than 0.060 inches. This white paper 
does not address liner designs with thicknesses equal to or greater than 0.060 inches. Rationale for liner thickness > 0.060 inch is 
unknown other than a functional rationale related to design pressure. 
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titled “Damage Tolerance Life Issues in COPVs with Thin Liners.” For example, current NDE 
inspection methods, prior to vessel autofrettage!®, consist of dye penetrant of the liner surface 
and X-ray volumetric inspection of the liner weld region. The typical "special" sensitive dye- 
penetrant method qualified per NASA-STD-5009 has a surface flaw detection limit of 0.050 inch 
long by 0.025 inch deep and X-ray inspection can detect a flaw of size 0.6t (where “‘t” is total 
material thickness the X-ray encounters). A few COPV vendors have implemented an improved 
“special” dye-penetrant method having a surface flaw detection limit of 0.030 inch long by 
0.015-inch deep. For a 0.030-inch-thick liner, D&DT analysis thus assumes the metallic liner 
contains a remaining ligament size ranging from 0.005 inch to 0.015 inch using typical or best 
dye-penetrant methods, respectively. Understanding the growth kinetics of small flaws shown in 
Figure 4.1 is extremely complex and predicting the life of thin liner COPVs will require 
extensive work before it is achievable. Appendix X3 of ASTM E647-15 (Ref. 10) provides test 
method guidelines where small or short fatigue cracks control a significant fraction of the 
structural life. 


Laser Notch 


Tensile Fatigue Test Coupon * as , = : PA 
Containing Simulated Surface = Ductile Fracture = 


je—0102 inch 


Flaw (laser notch) 


SEM Micrographs showing a partial through-the-thickness 
=0.023 inch fatigue crack. 


Note: EP - Elastic Plastic fatigue crack growth 
LE —Linear Elastic fatigue crack growth 


Figure 4.1. Test coupon results simulating the growth of a small surface flaw (crack) in a 
0.032-inch-thick metallic liner. The scanning electron microscope (SEM) micrograph (right) 
shows the through-the-thickness fracture surface depicted (red cross-section) in the drawing 


(left). (Ref, 22) 


Compounding the D&DT analysis is the fact that all liner flaw inspections occur prior to COPV 
autofrettage. In the autofrettage process, the metallic liner is plastically expanded into the 
composite overwrap. As a result of the liner expansion, a crack-like flaw contained in the liner 
can grow leaving a smaller remaining ligament that cannot be accounted for using conventional 
D&DT engineering tools because the remaining ligament is on the order of the grain size thereby 
violating continuum mechanics assumptions embodied in the existing tools. Such phenomena — 
cracks in yielded thin ligaments — are not unique to COPVs. In some cases, the influence of 
through-the-thickness plastic yielding is accounted for using a rule-of-thumb, or correction 
factor, on the LEFM solution; however, at certain length scales, such an approximation becomes 
inappropriate. Unfortunately, LEFM underlying assumptions are frequently misunderstood, 
misinterpreted, or possibly ignored. 


'6 Autofrettage is a metal fabrication technique in which the assembled COPV is subjected to internal pressure that significantly 
exceeds the operating pressure, causing the metallic liner to yield plastically. Reducing the autofrettage pressure results in 
internal compressive residual stresses in the metallic liner and tensile residual stresses in the composite overwrap. 
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In addition, thin liners are susceptible to highly localized permanent deformations resulting from 
liner expansion (embossing the overwrap interior surface onto the liner exterior surface). Such 
deformation processes have the potential to create regions of strain localization and influence the 
liner response when the autofrettage pressure loading is removed since local liner buckling may 
develop contributing to liner/overwrap disbond. Because the remaining liner ligament can be 
small, the embossing effect can have a further harmful effect by resulting in ligament plastic 
collapse, which is unrelated to classical fracture failure mode. The current traditional 
engineering analysis paradigm (discussed in the preceding section) may not apply adequate 
physics-based analysis tools and associated test methods that develop sufficient understanding 
for small material length scale designs. Currently, light COPV technology has surpassed the 
Agency’s ability to predict performance and thus to understand and manage flight risks. 


4.3.2 Example # 2 - Advanced Stirling Converter (ASC) Heater Head Fatigue/Crack 
Growth, April 2013, Tl-12-00829 


New and innovative designs constantly require using small length scales to increase component 
efficiency. An example of how innovative designs push our ability to predict D&DT is the ASC 
heater head design. The NESC conducted a technical assessment to review the life-prediction 
results for a nominal thin-wall (0.005 inch) heater head component. The design also required a 
cast MarM-247 component that contained large grains approaching the wall thickness dimension. 
It was found that the Agency and its contractor were using LEFM-based methods to predict the 
fatigue life of the component. The technical issue addressed was whether LEFM-based life- 
prediction methods can be used when a lack of similitude is indicated by accelerated small crack 
growth. Figure 4.2 shows that small (short) fatigue crack growth rates are accelerated compared 
to growth rates of long cracks. The NESC assessment concluded (Finding F-2) that LEFM is not 
capable of accurately and consistently predicting the fatigue crack growth (FCG) behavior of 
small cracks (i.e., small length scale cracks that approach the size of the microstructure). For 
large grain MarM-247 material, fracture-mechanics-based fatigue life-prediction methods must 
include either accurate FCG data established at the appropriate length scale or local stress and 
micromechanics models (Ref. 23) containing an understanding of the local damage processes 
depicted in Figure 4.3. It is important to note that using LEFM for this crack size to 
microstructure length scale does not meet the intent of standard practice of ASTM E647-15, 
Appendix X3.1 (Ref. 10). 
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Fatigue Crack Growth Rate — da/dN 


Crack-tip Stress Intensity Range - AK 


Figure 4.2. Comparison of fatigue crack growth rates for small (short) and long cracks where 
da/dN is the crack growth per loading cycle and AK is the difference between the maximum 
and minimum stress intensity factors. 


Figure 4.3a. Micrograph showing small fatigue crack that initiated within carbide inclusion 
and crack growth in large grains where @is the misorientation angle between two adjacent 
grains. The location of the carbide is shown using a circle (Ref. 23). 

Note: 25um ~ 0.001 inch. 
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Figure 4.3b. Micrograph showing small fatigue crack morphology (complex slip-band 
cracking damage process) in large grain (average grain size = 75 um, maximum grain 


4.3.3 


size = 390 ym) (Ref. 23). Note: 25um ~ 0.001 inch. 


Example #3 — Carbon/Carbon Silicon Carbide (C/C-SiC) — Material 
Characterization and Modeling — Phase II, May 2013, TI-11-00699 


The NESC conducted an extensive three-phase technical assessment to determine whether a new 
complex high-temperature ceramic matrix composite for the pintle and pintle guide parts could 
be certified for use in the Orion launch abort system attitude control motor (see Refs. 24 and 25 
for example). A primary objective of the NESC assessment was to understand the capability of 
the continuum-based modeling and analysis used to estimate design margin and investigate 
alternate modeling approaches to improve the predictive capability of the current continuum- 
based method. The NESC final recommendation was to not use the current continuum-based 
material modeling approach for analytically estimating design margin and to pursue a 
performance-based testing method that contributes to certification. The NESC found (Finding 
F-3, see Refs. 24 and 25) that while the continuum-based model may adequately estimate room- 
temperature break-test peak loads for specimens, it is not adequate to predict C/C-SiC pintle and 
pintle guide parts behavior during operation due to: 


a. 


Limitation of linear-elastic shear stress-strain assumption contained in the continuum 
model. 


Lack of local material modeling of a 4-directional composite system for local stress 
recovery relative to transverse isotropy assumptions for global response prediction 
(i.e., ignored local material length scales). 


Lack of material properties over a range of elevated temperature that encompass 
operating temperature range. 


Uncertainty that standard material property testing provided appropriate material 
characterization and damage modes. 


Insufficient understanding of damage initiation and growth mechanisms. 

Lack of analytical damage prediction models. 

Insufficient understanding of the role of local microstructural features near strain 
concentrators (geometric features such as threads, notches, etc.). 

Insufficient understanding of operating environments on material and structural 
responses. 
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The NESC further observed that accurate computation-based design margin for C/C-SiC pintle 
and pintle guide parts will be extremely difficult without a large uncertainty in the estimated 
design margin because of the following issues: 


a. Complex loading environments (usually dynamic). 

b. Strongly indeterminate or highly variable boundary conditions. 
c. Poorly understood or variable material behavior. 
d 


Limited availability of C/C-SiC material with which to determine the required 
statistically based material property allowables. 


@ 


Insufficient material property data at relevant elevated operating temperatures. 


f. Significant uncertainties in the operating environments over the C/C-SiC pintle and pintle 
guide parts (non-uniformity of external local environments like temperatures, loads, 
pressures, both static and dynamic). 


g. Lack of a verified materials model to calculate localized stresses and strains needed to 
estimate design margin determination. 


h. Lack of validated stress or strain metric(s) to estimate performance (i.e., point values, 
values averaged over a unit-cell volume, von Mises effective stress metric). 


When faced with similar computational limitations in determining design margin in the past, 
NASA has successfully relied on a comprehensive testing approach to show component 
reliability and robustness (refer to Ref. 24, paragraph 6.2 entitled A Review: Alternate 
Methodology for Establishing Structural Reliability). Such a testing program would: 


a. Obtain a thorough understanding of the expected failure modes of the component through 
integrated analysis and testing, to include testing to failure to demonstrate that the 
component does fail as anticipated. 


b. Demonstrate design robustness by qualification testing of components for durations 
longer than anticipated for worst-case operation, at greater than maximum expected 
operating conditions (e.g., loads, temperatures, pressures, etc.), and with known realistic 
defects, damage, or degraded sub-components. 


c. Screen for production deficiencies by acceptance testing of each and every flight 
component under higher than nominal conditions adequate to expose detrimental flaws or 
degraded properties, but below excessive conditions that will induce damage in 
nominally acceptable components. 


d. Utilize reliable NDE methods before and after all qualification and acceptance testing to 
detect any initial defects, the initiation of any new damage, and the propagation of 
detrimental damage conditions. 


Although this approach has proven to be effective, such testing programs are very expensive and 
only apply to the specific material, structural configuration, and loadings considered. However, 
such programs do mitigate risk for the given design and application, thereby providing decision 
makers with credible performance assessment and guidelines. Just as inappropriate modeling 
and analysis pose potential hazards, so can a non-applicable test contribute to an inaccurate 
assessment. Diligence and technical rigor need to anchor our decisions and rationale for both 
testing and analysis. 
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4.3.4 Example #4 - STS-133/External Tank (ET)-137 Intertank (IT) Stringer Cracking 
Issue 


During the filling of the ET-137 with cryogenic liquid fuels for the last mission of the Space 
Shuttle Discovery (STS-133), cracking in the foam on an IT panel near the liquid oxygen tank 
interface was observed (see Ref. 226). Upon further inspection, it was discovered that not only 
was the foam cracked, but also two external hat-shaped stringers under that foam on the IT had 
long cracks along the stringer feet between the fasteners and the stringer sidewall. Detailed three 
dimensional (3D) FEMs were developed and analyzed to investigate the stringer elastic-plastic 
response and to assess the potential for local failure to develop in the stringer feet. 


Most structural materials contain some degree of anisotropy at the structural level (large or 
macro-level length scales) and/or at the material level (small material length scales). When 
understanding fracture-related performance is critical, materials models must represent both 
anisotropic behavior and material-specific failure mode attributes associated with multiple 
material length scales. For example, some material orientations are susceptible to strain 
localization based failure modes because of low fracture properties related to key microstructure 
features (small material length scales). This common failure mode is known to occur in 
composites and some monolithic metallic materials. In both cases, strain localization can occur 
at interfaces. In metallic materials, these interfaces are grain boundaries; and in composites, these 
interfaces are interlaminar (delaminations between lamina) or intralaminar cracking 
(delamination migration pathways through lamina). For these specific cases, strain is localized 
at the interface thus reducing fracture properties and/or fatigue life. These important material 
specific and anisotropic characteristics are extremely difficult to model analytically and to 
characterize by test across different material length scales and yet they are extremely important 
in assessing D&DT performance and understanding risk. 


Most metallic sheet and plate products contain some degree of material anisotropy due to the 
fabrication, machining, and forming processes that is generally small. However, aluminum- 
lithium plate material can exhibit significant material length scale dependencies due to a high 
degree of anisotropy along with grain boundary strain localization effects. These complex 
material attributes influenced by fabrication and heat treatment cannot be suitably predicted 
using the conventional engineering analysis method discussed in Section 4.1 and material 
characterization testing is needed to support the design. Selecting an alternate material system is 
often based strictly on a weight-savings argument without understanding the potential risks being 
assumed if only limited test data are available. 


Based on the detailed Space Shuttle ET IT stringer cracking assessment findings (Ref. 26), it was 
recommended (recommendation R-1) that aluminum-lithium alloy 2090 T8 sheet material should 
not have been used as a stringer material, because, 


e The stringers can develop complex, localized through-the-thickness bending and shear 
stresses near the fasteners due to fabrication process, assembly loads, and combined 
mechanical and thermal loading. 


e The through-the-thickness properties of alloy 2090 T8 are marginal (i.e., low ductility) 
and material processing variations, even within specification, can alter microstructural 
properties and unknowingly exacerbate cracking prior to ultimate strength under stringer 
local environments exhibiting bending and shear stresses. 
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At present, and again similar to the Space Shuttle ET IT stringer issue, the aft bulkhead of the 
Orion Exploration Flight Test (EFT-1) capsule, constructed of an aluminum lithium alloy, 
cracked during proof testing and is also an indication that current standard engineering practices 
have difficulty in predicting material local-property-related damage-tolerance performance and 
that previous recommendations are not lessons learned. Conversely, the analysis effort did not 
adequately represent the uncertainty in the predictions and did not account for these localized 
influencers (i.e., material length scale influences). 


Unfortunately, in 1966, long before the Space Shuttle ET stringer issue, similar cracked stringer 
failures were noted, investigated and reported as a lesson learned in 1971 for the Saturn S-II-4 
skin-stringer assembles built using 2020-T6 extrusions (Ref. 27). The final paragraph states: 
“The major lesson learned from the 2020 cracking problem is that new materials require more 
than the usual specification verification tests for chemical and mechanical properties. 
Additional testing efforts must be expended to provide guidelines for satisfactory handing and 
fabrication procedures. Materials with adverse transverse properties should be especially 
suspect and signal a need for special attention.” Apparently, it was a lesson not learned. 


4.3.5 Example #5 - D&DT of COPV Composite Overwrap 


The second COPV primary risk is associated with the composite overwrap and the ability to 
predict composite damage and perform life assessment for the COPV (see Appendix C and Ref. 
28). The composite overwrap carries a significant portion of the internal pressure loading 
imposed during autofrettage, proof testing, and operation. For space flight systems, these 
COPVs are deployed to achieve a long-term service life, with a situation for which there is 
minimal if any opportunity for inspection and repair. One failure mode is the stress rupture of 
the composite overwrap caused by local material failure of the fiber in tension. Such a failure 
involves multiple micromechanics effects including fiber failure, fiber/matrix interfacial failure, 
redistribution of local loads, and through-the-thickness damage mechanisms — all predominately 
small material length scale mechanisms, which exceeds our ability to observe or inspect. 
Although rare, such a failure after the COPV has been in service for a period of time is 
potentially catastrophic. 


The NESC performed an assessment on carbon/epoxy stress rupture (Ref. 28) and found that the 
models typically used by industry for estimating the threat of stress rupture were not supported 
by an adequate set of experimental data and a comprehensive test program was recommended. 
Issues with complex pressure-time history, uncertainty treatment, stress ratio determination, and 
available data have not been adequately resolved in past modeling efforts. Current and future 
NASA programs rely on these data for estimates of COPV reliability, but these reliability 
estimates cannot be determined with confidence. 


The interactions between the liner and the composite overwrap during fabrication and proof 
testing challenge the current engineering analysis tools in assessing potential defects, disbonds, 
inelastic response, and local imperfections. Further, the applicability of fiber failure models 
based on fiber strand tests have not been validated in an overwrap application where fiber strands 
and matrix interact, and local load paths develop. Reliance on industry design standards and 
historical COPV performance data provide the standard engineering practice to-date until 
validation tests are defined and performed and used to validate new analysis models at the 
appropriate material length scales. Such a validation approach was expressed by the alternate 
opinion in Ref. 25 and led to the following recommendation: “The failure mechanism(s) 
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underlying stress rupture of COPVs is not understood, and serves as the barrier to reliable 
predictive models and safe COPV operation. A mechanistic model of COPV stress rupture 
based on damage evolution data obtainable from transverse tensile tests has a greater likelihood 
than the proposed empirical approach for achieving a reliable predictive capability, and in a 
shorter time and at much lower cost.” 


4.3.6 Example #6 -- Pluto New Horizon Launch Vehicle Tank Qualification Test Failure; 
Final Report, Vol. I, NESC Report RP-06-02-05176, 08-28-2008 


This qualification test failure was attributed to a failure at the end of a stiffener flange near an 
attachment point. Finding F-3 states that the structural analysis and margin assessment were 
incomplete. The global FEMs were unable to properly model or converge when the anisotropic 
elasto-plastic material properties of the tank were considered. Local spatial dependencies 
included local changes in curvature, presence of loaded bolt hole, and changes related to stiffener 
termination. Since the directional dependency of the material properties at multiple material 
length scales (macro and micro) was an essential part of the crack initiation, propagation, and 
rupture process, the inability of the analysis model to converge numerically meant the 
qualification test failure could not have been predicted by the global model, but also that the 
global model could not be used to determine the capability to withstand the expected ground 
processing and flight loads related to the Pluto/New Horizons mission. Local analysis models 
were formed from a subset of the global FEM (i.e., using a global-local approach) and were able 
to converge numerically. However, these local models were based on the same material 
modeling approach as used in the global model and hence, their results were similarly limited in 
the modeling of the anisotropic nonlinear material behavior at the appropriate material length 
scale. Consequently, their results could not be relied upon to determine the actual stress state of 
the material in the region of failure where small material length scales influence the response. 


4.3.7 Other Examples 


Other examples from previous NESC assessments can be cited to illustrate this issue and its 
pervasive nature throughout the Agency, and in some respects, throughout the broader aerospace 
industry. These additional examples where multiple material length scales are involved include: 


e Reentry thermal protection system — limited material data, complex local multi-physics 
environments, lack of fiber bridging material modeling, lack of fracture data for 
substrate, coating and interface conditions, coating spallation. See Ref. 29. 


e Margin assessment of adhesive joints for electronic components — lack of material 
property data, lack of material model through a glass transition region, reliance on linear 
elastic stress analysis results within a nonlinear material response regime. 


e Margin assessments for nonlinear systems using ratios of analysis results in lieu of a 
material allowable and over reliance of solid-geometry-based finite element meshes as 
‘high-fidelity modeling’ — meshing and modeling are intrinsically different. 


e Using LEFM for non-homogeneous, nonlinear closed-cell foams assuming a linear 
elastic, homogeneous, continuum-like material response (see Figure A-5 of Ref. 30). 


e Characterizing new materials and the supporting analyses based on limited material 
availability, conventional continuum mechanics assumptions, and uncertainty in 
environments and loads resulting in severe consequences if failure occurs. 
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e Increased dependency on material length scale for advanced manufacturing techniques 
compared to traditional material processing and fabrication. Changes in damage 
mechanisms and growth for additive manufacturing processes. Developing appropriate 
material models for new fabrication and manufacturing processes. 


e Propulsion valve bellows seals must carry pressure loads as reliably as other pressurized 
propulsion system components including tanks (metallic or COPV), pipes, and valve 
bodies. To meet valve design criteria, the bellows are often hydro-formed (plastically 
deformed) from multi-ply, thin-wall tubing material where the through-thickness 
dimension is on the order of the metallic grain size. Valid through-thickness material 
properties, damage-tolerance prediction methods, and certified non-destructive inspection 
techniques do not exist to evaluate or certify these fracture critical components for flight. 


In some cases, lack of or limited material data was the primary issue; in other cases, there was a 
reliance on existing methods because of Engineering familiarity and possibly schedule or cost 
pressure. However, the real culprit is our inability or unwillingness to acknowledge and accept 
limitations in our understanding and our desire to develop an engineering solution even if it may 
be inappropriate. Often, in these cases, we have been protected from ourselves because of 
previous mandated factors of safety. Thus, as we extend these designs and life-extension 
predictions are made, we unknowingly may be increasing risk causing the unknown unknowns to 
become critical players in the D&DT evaluation. 


Because of Agency funding constraints, it has become increasingly apparent that Engineering 
may be over relying on existing computational methods driven in part by their ease of use and 
the availability of significant computing power to solve large-scale finite element meshes. 
However, the Agency needs to develop a sufficient technical understanding of the capability and 
limitations of these tools by conducting thorough testing and by proactively developing advanced 
engineering computational tools with validated predictive capabilities for anticipated advances in 
materials and design capabilities. The current design environment and engineering practice for 
high-risk missions involving severe or even hostile environments require a new level of 
accountability on the analyst to understand the technical basis of the tools, their assumptions, 
their limitations, and their applicability to the current design problem across multiple material 
length scales. 


5.0 Needs and Recommendations 


This report has identified several deficiencies in the Agency’s standard Engineering practice 
related to D&DT and provided examples from previous NESC assessments that illustrate the 
need for re-tooling. Three needs have been identified and are described in the following sections 
followed by a list of recommendations. 


5.1 Understanding the Limits of Engineering Methods 


An important weakness in the Agency’s ability to develop new technologies lies within our 
ability to conduct proper trade studies. As new missions are formulated and advanced human 
space flight vehicles are developed, Engineering must also understand the limits of its predictive 
tools; in particular, those used for trade studies. Traditionally, trade studies have been performed 
using rapid design tools with system-level requirements without acknowledging and accounting 
for potential influences of D&DT that may redirect a trade study. Engineering must ask the 
difficult questions like, can we predict the D&DT capability of inflatable habitats? How long 
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can we safely operate vehicles in deep space? Do we need new predictive tools for advanced 
materials and advanced manufacturing? Many other questions can and should be asked. For safe 
human deep space travel, the Agency cannot rely on the current LEO reliability method that is 
based primarily on repair and rapid/reliable return to earth. As the Agency moves toward long- 
duration, deep-space, human exploration, the D&DT engineering approach must be able to be 
proactive and develop new tools where vulnerabilities are as yet unidentified, but critical for 
mission success. Such an approach is anticipated to require a workforce having a different skill 
set that asks the difficult questions, challenges conventional wisdom, and pursues technical rigor 
above all. 


5.2. Need for Advanced Engineering Computational Tools 


The erosion of the Agency’s engineering D&DT computational capability has not occurred over 
night. The examples listed in the previous sections show that the engineering community has 
had difficulty in developing rigorous understanding and methods to assess risk when complex 
material modeling issues were not recognized, understood, or addressed properly. These 
difficulties will be compounded as designs require a multi-physics modeling approach (e.g., the 
frangible joint assessment, additive manufacturing, and multifunctional materials). The Agency 
has reached a point where new spacecraft design requirements have made our D&DT predictive 
capabilities obsolete in areas when an understanding of complex materials and local 
environments is required. The Agency’s engineering and research communities have understood 
our weaknesses for many years and have proposed solutions based on new CM and multi-physics 
based technologies to develop advanced engineering predictive D&DT tools (see Appendix D). 
Nearly four years have passed since the Fundamental Engineering Sciences initiative and 
advanced computational-materials-based methods were proposed to address vulnerable 
engineering D&DT practices. The Agency must become proactive and develop new predictive 
engineering tools applicable to emerging material systems and manufacturing technologies to 
ensure safe, deep-space, human and robotic exploration and interplanetary habitation. The 
Agency needs to learn the lesson from Saturn (Ref. 24) that “new materials require more than 
the usual specification verification tests for chemical and mechanical properties.” 


5.3 Validation Testing Needs 


With Agency funding constraints, it has become increasingly apparent that engineering may be 
overly reliant on existing analysis methods and not developing a sufficient technical 
understanding by conducting a thorough testing approach including sufficient replicas to 
understand variability. Validation tests generate data for the purpose of model validation and are 
defined and developed through cooperative and collaborative efforts between the test engineer 
and the analyst — usually they are different individuals. A series of such validation tests may be 
performed as the design progresses through a building-block development process (i.e., replica 
tests, and/or specific feature tests). Through the verification and validation (V&V) process, a 
model can be declared “validated for its intended use” once it has met the validation 
requirements. These requirements are established in the V&V plan and often include 
descriptions of specific tests that need to be run or other referents (i.e., data, theory or 
information) that need to be gathered, what specific measurements need to be acquired from the 
tests and accuracy statements about the agreement between simulation outcomes and 
experimental/test outcomes, as discussed in Appendix E. 
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Because of the complexities associated with local environments that drive important small 
length-scale damage processes and their influence on D&DT, standard test methods often do not 
capture the proper and complete understanding required to develop material models reflecting 
multiple length scales. New test methods must be developed and introduced into the standard 
engineering practice to verify and validate complex D&DT analysis findings. New NDE 
inspection techniques must be developed and validated to ensure structural integrity and to 
characterize the local material state and its evolution as the environments and operational loads 
are imposed. 


5.4 Recommendations 


The recognition of the need for and pursuit of advanced engineering practices related to new 
exploration missions is a critical step to ensure that each Agency technical discipline is prepared 
for the future. The Agency must acknowledge that new standard engineering practices are 
required to exploit the emerging materials, fabrication techniques, and design configurations 
needed to meet the life-performance requirements placed on new spacecraft system. This is 
especially true for human exploration missions. Mission success and safety require technical 
rigor across multiple material length scales when assessing D&DT. A multi-discipline CM 
based effort is recommended with the Materials Technical Discipline Team (MTDT) having the 
lead with active participation and collaboration with both the Structures TDT and the NDE TDT. 
It is recommended that this effort include the following items as a starting point. 


e Provide significant and sustained funding for Agency-sponsored CM research and 
development leading to a re-tooling of the D&DT standard engineering practice. 


e Generate a report cataloging the current state-of-the-art of CM engineering across the 
aerospace community, especially within the Agency. This report should identify 
potential partners from other government agencies, in particular the DoE and DoD, for 
developing integrated computational material engineering (ICME) and to foster 
interagency communication and technical exchange. 


e Establish an NESC multidisciplinary team (MTDT led with Structures and NDE TDTs) 
to develop CM-based engineering D&DT test and computational methods. Exploit the 
NESC leadership structure and technical focus in providing oversight and accountability. 
The following recommendations should be focused on specific material systems, 
fabrication techniques, and local environments important to the Agency for future 
spaceflight missions. 


e Establish a work effort to address near-term thin-liner D&DT issues associated with 
lightweight COPV methodology needs. These needs will likely include validation testing 
(properties, component and NDE test development) and computational methods 
development. 


e Establish a work effort to identify D&DT needs and issues associated with components 
fabricated using additive manufacturing techniques. These needs will likely include 
validation testing (properties, component and NDE test development) and computational 
tools development. 


e Develop testing procedures and techniques to generate test data to validate new D& DT 
strategies and procedures focused on small material length scales. 
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e Assess the current and develop new NDE capabilities to support validation testing 
focused on small material length scales. 


e Develop and implement new standard engineering practice D&DT guidelines and tools 
applicable across multiple material length scales for the future aerospace systems. 


e Increase awareness of material length-scale influences on D&DT related to standard 
engineering practices using a workshop/forum/NESC academy approach. 


e Educate the Agency workforce regarding material length scales and their impact on 
D&DT assessments and the potential limitations of the standard engineering practice 
using a short-course format. 


6.0 Concluding Remarks 


With our increased reliance on analysis methods to assess designs, it is critical that we recognize 
the potential shortcomings of our current standard Engineering practice tool set and the need to 
re-tool. This re-tooling process involves developing an understanding of the input data 
requirements of the computational tools, recognizing the limitations of current computational 
tools, developing the appropriate tests to conduct where important understanding is needed to 
validate the computational tools, and investing in new engineering tools that will be needed for 
new designs, new materials, and/or new fabrication processes to meet the ever-increasing 
demands of more ambitious space exploration. Since Engineering likely spends most of their 
time under the gun to produce new designs under cost and schedule constraints, they likely do 
not have the time or motivation to stay abreast of the new advancements in technology that will 
be required. Sustained investments in skilled people placed in a cooperative environment where 
multidisciplinary R&D can be pursued is what is critically needed for progress in these areas — 
including elements of advanced computational methods, micro- and meso-mechanical modeling, 
materials science, and advanced experimental methods. Only in such an environment can 
physics-based micro-mechanics and meso-mechanics analytical models be integrated with large- 
scale computational capabilities, and rigorously validated. To make progress on a reasonable 
time scale, NASA’s multidisciplinary team and Agency-wide capabilities need to be linked, 
either formally or informally, to capabilities that exist at selected universities and research 
institutes to effectively incorporate the underlying micro-mechanical and meso-mechanical 
analytical modeling that needs to be developed and applied. Such physics-based modeling is 
required to achieve the understanding that is cited throughout this report. 


In addition, the Agency must take a reasonable/realistic view of new designs relative to current 
engineering analytical capability and limit challenging designs either when appropriate 
computational tools are not available or not verified, or when limited amounts of material and/or 
parts are available to understand mechanical and thermal response, or when uncertainties with 
regard to bounding environments are high. Otherwise, the Agency must invest in developing a 
more thorough testing approach to mitigate the risk associated with the lack of knowledge and 
understanding for a given design and mission and to acknowledge the limitations of the 
computational tools to solve all problems. 
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Appendix A: Material Length Scale Description 


Various definitions have been given for the various material length scales as they relate to 
mechanical design [13]. Following the definitions of Mishnaevsky [31], macro-scale refers to a 
specimen of coupon or structural element or even vehicle level analysis model where the length 
scale is on the order of more than a millimeter. Material response at this level is analyzed using 
conventional continuum mechanics and loosely defined as the structures discipline. Micro-scale 
refers to a microstructural level on the order between | micrometer and | millimeter. Material 
behavior at this scale falls into the area of material science, and is analyzed using methods of 
both physics and mechanics of materials, including micromechanics and fracture mechanics. 
Loosely defined, this scale is the materials discipline. Nano- and atomistic scales are less than | 
micrometer. Material behavior at this scale falls into the area of the physics of the materials and 
is loosely defined as basic physics. 


Micro-scale models essentially imply that nano-scale effects do not need to be considered. 
Likewise, macro-scale models assume that microstructure effects do not need to be considered. 
In all cases, finite element techniques have been and are being used to solve the various partial 
differential equations associated with the fundamental conservation laws. 


In addition, some researchers use the term “meso-scale” to signify a scale “in between” any two 
of the three previously mentioned scales. Herein, meso-mechanics studies quantitatively the 
interaction and synergistic effects of many elements of microstructures (as inclusions, voids, 
shear bands, microcracks — general heterogeneity of the material) on strength and mechanical 
properties of materials. Mesoscale, per Needleman [7], then defines an intermediate scale 
between direct atomistic and an unstructured continuum description of deformation processes. 
While the meso-scale definition is somewhat arbitrary, meso-scale representations typically 
consider some aspects of both discrete and continuum influences (e.g., discrete dislocation 
plasticity where dislocations are explicitly simulated within a localized homogenized elastic 
continua). 


Consequently, various mechanics formulations provide a roadmap to address different material 
length scales and their underlying assumptions. Understanding the different length scales and 
selecting the appropriate mechanics formulation validated by test is crucial in developing 
advanced CM models (see Refs. 13 and 31). Evolving such science-based CM tools into 
engineering D&DT tools that are adopted as standard practice will need emphasis. 


The following figures depict critical length scale features related to crack growth in metallic 
materials (see Figure A-1) and important features that control damage in laminate composite 
materials (see Figures A-2 through A-4). The anisotropic nature of a foam’s microstructure is 
shown in Figure A-5. These figures are provided to indicate the importance of material length 
scales for different material forms under some characteristic length scale associated with a 
material feature at some level. These examples illustrate the local complexities and imply their 
increase as local environments become multi-dimensional. 
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Example: Metallic Materials 


(a) WHEN 
a=10mm 


LOW STRESS-LONG CRACK GROWTH 

LEFM - Linear Elastic Fracture Mechanics 
A Typical Stage II “tensile“ crack 

Crack tip plasticity difficult to observe 
Ratio a/rp high (>>1) 


a > 100 grains 


(b) WHEN 
a= 0-1mm 


a <10 grains 


(c) WHEN 
a=24m 


a <igrain 


Figure A-1. Three regimes of fracture mechanics with crack sizes given in relation to the 


HIGH STRESS - PHYSICALLY SMALL CRACK 

EPFM - Elastic Plastic Fracture Mechanics 
A Stage II "tensile" crack 

Crack and plastic zone readily observable 
Ratio a/rp of intermediate values. 


HIGH STRESS - MICROSTRUCTURALLY SHORT CRACK 


MFM - Microstructural Fracture Mechanics 
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and (c) MFM, Ref. 32. 
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Example: Laminate Composite Materials 


PROGRESSION OF FAILURE 


MATRIX DAMAGE ACCUMALATION AND INTERACTION 


T I 
ae Crack” Delamination 
ry at Crossed Cracks Free 


= Edge 
Delamination 
‘INTERLAMINAR MATRIX 


ss 
COMBINED 
O° Ply INTRALAMINAR AND 
Split INTERLAMINAR MATRIX 


INTRALAMINAR MATRIX 


LOCAL STRESS CONCENTRATION TO FIBERS 


LOCALIZED FIBER-MATRIX 
FIBER FRACTURE DISBOND 


[| = | 


CATASTROPHIC 
FIBER FAILURE 


Figure A-2. Failure mechanisms for laminates loaded in tension, Ref. 33. 
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Figure A-3. Failure mechanisms for laminates loaded in compression, Ref. 33. 
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_ Composite Damage Modes: Does 


Figure A-4. Important micro-scale defects that influence damage modes, Ref. 34. 
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Figure A-5. Foam anisotropic microstructure, Ref. 30. 
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Per Communications: Email (Attachment), Subject: ISS COPV Listing, From: L. Grimes- 


Appendix B: ISS COPY Listing 


Ledesma, To: R. Piascik, C.C.: M. Kirsch, dated 6-6-2016. 
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Name MFG Pressurant Part # Type Dia (inch) |Length (inch)|_ Liner Material /Liner Thickness MEOP 
ISS PCA (HPGT) Arde O2orN2| D4467 Spherical 37,9 Cryoformed 301 0.093 3400 
ISS Safer II Arde O2orN2| D4616 Cylindrical 6.56 8.93 Annealed 304L 0.05 10,000 
ISS NTA GD N2| 1F96201 | Cylindrical 19.7 45.23 6061-T6 0.05 3000 
ISS CIPAA Cobham Cylindrical 44 9.64 6061-T6 0.07 4500 
ISS PCU Arde Xe} D4636 Spherical 15.35 Cryoformed 301 0.03 3000 
ISS BTF SCl 1273084 | Cylindrical 44 12.1 6061-T6 ? 3000 
STS MPS Arde He) D4554 Spherical 26 Cryoformed 301 0.046 4500 
AMS-2 (Xe) Xe, He, or CO2 
VCAM He 
GBU (CO2) CO2, He, or Ar 
GBA Ar 
VGA Air 
SpaceDRUMS 
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Appendix C: Review - Variance to Reduce COPV Proof Factor from 1.50 to 
1.25 


R. Piascik (February 10, 2016) 


Based upon a request from Tim Wilson (Director, NESC), a rapid review was conducted to 
understand the technical rationale that justifies a composite overwrapped pressure vessel 
(COPV) proof factor reduction from 1.5 to 1.25 prompted by a variance request. 


Background: 


[A] Company is building carbon-fiber overwrapped COPVs that meets their current requirement 
of 2.0 burst factor and 1.5 proof test factor for all human spaceflight applications per NASA STD 
5019 (see Attachment 1) and ANSI/AIAA S-081A (see Attachment 2). [A company] has 
requested a variance on the NASA STD 5019 proof test factor requirement to allow for a 
reduction in proof factor from 1.5 to 1.25 in accordance with JSC 65828, refer to Attachment 3. 


NOTE: NASA STD 5019 paragraph 4.1.2.1.3 states that COPVs shall comply with the latest 
revision of ANSI/AIAA Standard S-081, refer to Attachment 1. ANSI/AIAA S-081A paragraph 
5.1.2 states COPV’s shall be proof tested to a minimum pressure of: P = /2(1 + Burst Factor) x 
MEOP (for a burst factor less than 2.0) or =1.5 x MEOP for a burst factor equal to or greater 
than 2.0, refer to Attachment 2. (MEOP = [Maximum Design Pressure] (MDP) per the 
requirement of NASA STD 5019). 


Discussion: 


The following is a list of questions directed at specific issues associated with developing a 
rationale for lowering the proof pressure requirement from 1.5 to 1.25. 


1. JSC 65828, Attachment 3, contains the proof pressure requirement of 1.25. What is the 
technical rationale for the JSC 65828 requirement for a 1.25 proof pressure limit? This is 
an important question because JSC 65828 is a primary document (standard) that allows the 
reduced proof which is in conflict with the governing NASA STD 5019 standard for fracture 
control of spaceflight hardware. The NASA STD 5019 requires COPV’s to meet ANSI/AIAA 
S-081-2000 and therefore all COPV’s with a design burst factor of 2.0 must meet the proof 
pressure requirement of 1.5. 


Based on discussions with L. Grimes-Ledesma and Ivatury Raju, the JSC 65828 proof test 
requirement evolved in 2006 from the Constellation Program (SDVR CxP 70135) who lowered 
the 1.50 proof test factor for COPVs to 1.25. A listing of the Programs that use a proof test 
factor is shown in charts entitled “Composite Overwrapped Pressure Vessel Proof and Burst 
Factor Analysis Result” by Nathanael Greene, JSC ES Pressure Systems/Fracture Technical 
Discipline Lead, Document 02032016, January 3, 2016. The discussions revealed that proof test 
factor reduction was based on concerns for composite overwrap damage caused by 

autofrettage. Note: Autofrettage is a sizing pressure operation where pressure driven deflection 
is used to plastically yield the metal liner into the overlying composite in order to induce initial 
stress states in the metal liner and composite. The autofrettage parameters are governed by the 
proof pressure factor. Therefore, a proof pressure factor of 1.5 will have a higher autofrettage 
sizing pressure than a proof factor of 1.25. The higher autofrettage sizing pressure mandated by 
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the 1.5 proof pressure factor is the root of the concern for composite overwrap damage from the 
stress rupture reliability calculations. 


In addition, the Nathanael Greene charts suggest a potential conflict between NASA Structures 
requirements and NASA Fracture Control requirements (Chart 2). Chart 4 indicates that non- 
conformance requests are prevalent. While the pressure vessel standards and fracture control 
requirements have not changed, the structures requirement changed from 1.5 to 1.25 proof test 
factor (Chart 6). However, technical rationale for the proof test factor reduction has not been 
identified in this review. 


2. What data suggests a 1.5 proof pressure factor produces unacceptable composite damage? 
As part of addressing this question, understanding of past history because the entire space COPV 


community has used and continues to use the existing ANSI/AIAA S-081-2000 requirement of 
1.5 proof pressure factor. The 1.5 proof pressure factor has been used since 1984. No historical 
insight (data) has been given relative to the experience of using the 1.5 value. It is therefore 
presumed that no significant operational or manufacturing issues have negated its use of 1.5 
proof pressure factor as an industry-wide requirement. 


The JSC 65828 document is a primary structures standard that requires all COPV to have a 
design burst pressure of 2.0 and a proof pressure factor of 1.25. The rationale for the lower 

1.25 proof factor is based on stress rupture concerns during autofrettage as calculated by a 
carbon fiber breakage model. The model calculates a fiber strain value that approaches ultimate 
strength for a 1.5 proof factor based autofrettage. The fiber strain calculation suggests that 
autofrettage could produce unacceptable fiber damage leading to a reduction in stress rupture life 
and decreased reliability. The NESC has and continues to investigate the fiber breakage model; 
the model has not been validated by test and has been characterized by an NESC finding as 
having “significant shortcomings,” per NASA/TM-2012-217564 (NESC-RP-09-00537). It is 
also worthy to quote the alternate opinion published in this NESC document; “Bottom line: The 
failure mechanism(s) underlying stress rupture of COPVs is not understood and that is the barrier 
to reliable predictive models and safe COPV operation.” 


Other concerns have also surfaced relative to metallic liner pressure produced strain during a 
1.5 proof pressure derived autofrettage. It is thought that excessive autofrettage tensile and 
compressive strain could be produced causing liner damage and buckling. 


3. What methodologies are available to ensure acceptable COPV performance? 


a. Burst testing: Currently a 2.0 burst test factor validates the design requirements during 
qualification phase of COPV design development. Once the COPV design is set and 
validated by burst test, no further burst testing is conducted during the life of the design. 


b. Proof testing: NASA STD 5019 defines proof as a load or pressure in excess of limit 
load or the MDP by a defined factor applied to a structure or pressurized hardware to 
verify structural acceptability or to screen flaws. Therefore, each COPV is proof tested 
as the last step for final acceptance. Currently, the final verification of carbon fiber 
integrity after proof testing is by visual inspection. When lowering the proof pressure 
factor by 50% (1.5 to 1.25), what data are we obtaining that /ensures that the product 
quality has been maintained? For example, is the lower proof test factor screening for the 
same defects? Can the lower proof factor allow for subtle (unknown) changes in the 
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manufacturing process that pass proof testing and then affect operation? There seems to 
be no published consideration of the ramification of lowering the proof test factor. 


NDE for fiber breakage: Currently, there is no qualified NDE method for the detection 
of fiber breakage other than outer surface visual inspection. This is important, because 
fiber breakage during autofrettage and proof testing is only noted by outer surface visual 
inspection. 


NDE for metallic liner damage: NDE methods are available to detect liner damage after 
proof/autofrettage. Laser profilometry measurements have been demonstrated for 
detection of liner deformation (buckling) after autofrettage. Upon completion of a 
probability-of-detection (POD) study, an eddy current method will be available for use 
for the detection of liner cracking after proof testing. 


4. What alternate approaches have been suggested to mitigate the composite damage concern 
resulting from fiber breakage? No alternative strategies have been noted during this rapid 
review to mitigate fiber breakage. If the current fiber-breakage model is used to rationalize a 
proof test factor/autofrettage pressure, it could also be used to rationalize added carbon fiber 
overwrap to low fiber strain to an acceptable level. Obviously, added carbon fiber overwrap will 
result in increased weight. The results of a valid trade were not found that weighs the risks/costs 
of lower proof versus alternatives. 


Summary 


This review suggests the lowering of the COPV proof factor is based on: 


1. 


Use of a fiber breakage model that is not validated by test and was characterized as 
having significant shortcomings. 


Lack of historical data as to the successful use of 1.5 proof pressure factor. 


Lack of justification as to how the lower proof test factor affects verification of structural 
acceptability or to screen for flaws. 


Lack of any trade studies as to other alternate approaches to mitigate autofrettage effects. 


Substantial number of non-conformances related to structure and fracture control proof 
factor requirements with a lack of available technical rationale have been identified. 


It is highly recommended that a thorough independent assessment be conducted to understand 
COPY state-of-the-art as it relates to proof test and autofrettage requirements. 
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ATTACHMENT #1 
NASA-STD-5019 Fracture Control Requirements for Spaceflight Hardware 


1.2 Purpose 


Fracture control is implemented to reduce the risk of a catastrophic failure from a defect or 
damage. The intent of this standard is to provide fracture control requirements for spaceflight 
hardware. A variety of fracture control considerations and options are addressed, some of which 
may not be applicable to a given design. Information is provided to assist the user in the 
development of an effective Fracture Control Plan and other fracture control documentation. 


4.1.2.1.3 COPVs 
COPVs shall comply with the latest revision of ANSI/AIAA Standard S-081, Space Systems- 
Composite Overwrapped Pressure Vessels (COPVs), with the following tailoring: 


a. MDP* shall be substituted for all references to MEOP (Maximum Expected Operating 
Pressure). 


b. Mechanical damage control shall include protective covers and damage indicators as a 
minimum unless otherwise approved by the RFCB [Responsible Fracture Control Board]. 


c. If damage indicators are utilized, the indicator shall be inspected between missions. 


*Maximum Design Pressure (MDP): For a pressurized system, the highest pressure defined by 
the maximum relief pressure, maximum regulator pressure, maximum temperature, and transient 
pressure excursions based on two credible system failures. 


4.2.4.4 Flaw Screening for Fracture-Critical Parts 

a. Fracture-critical parts shall be screened for flaws by NDE, proof testing, or process control. 
b. RFCB approval shall be required for flaw screening by proof tests or process control. 
4.2.4.4.1 NDE 


a. NDE shall be done on fracture-critical parts to establish that a low probability of preexisting 
flaws is present in the hardware. 


b. NDE inspections for fracture control shall be performed in accordance with NASA STD-(1)- 
5009 for metallic components and meet the intent of MIL-HDBK-6870 for composite 
components. 


c. Hardware that is proof tested as part of its acceptance (i.e., not screening for specific flaws) 
shall receive post-proof NDE at critical welds and other critical locations identified in the 


Fracture Control Plan. 
4.2.4.4.2 Proof Test* 


a. Prior approval shall be required from the RFCB when a proof test is used as the flaw screening 
technique. 
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b. Documented rationale shall be provided, demonstrating the component is not expected to 
experience significant crack growth during the proof test, and/or a presumed crack size after the 
proof test adequately accounts for growth during the test and demonstrates adequate damage 
tolerant life. 


c. When it is judged that a proof test is appropriate to screen a component or structure for flaws, 
the proof test shall occur at the in-service temperature and environment. 


* Proof Test: A load or pressure in excess of limit load or the MDP by a defined factor applied to 
a structure or pressurized hardware to verify structural acceptability or to screen flaws. 


4.2.4.4.3 Process Control 


a. Prior approval shall be required from the RFCB (Responsible Fracture Control Board) when 
process control is used to determine the initial defect sizes for damage tolerant analysis and/or 
testing. 


b. Process control rationale submitted for RFCB approval shall include a statement explaining 
why this alternate approach is being applied, an overview of the hardware, the manufacturer’s 
experience base, process control during manufacture and subsequent life of the component, all 
component testing, and summary arguments. 
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ATTACHMENT #2 


ANSI/ATAA S-081-2000 Space Systems — Composite Overwrapped Pressure 
Vessels (COPV) 


5.1 Acceptance Test Requirements 


Acceptance tests shall be conducted on every COPV to verify workmanship and identify 
manufacturing defects. Accept/reject criteria shall be formulated prior to tests. The test fixtures 
and support structures shall be designed to permit application of all test loads without 
jeopardizing the flightworthiness of the test article. As a minimum, the following tests are 
required: 


(a) General inspection per Section 4.5.1, 
(b) Proof pressure testing, 

(c) Leak testing. 

5.1.1 Non-Destructive Inspection 


Every COPV shall be subjected to visual and other non-destructive inspection (NDD, per the 
inspection plan of Section 4.5.1, to establish the initial and post-proof condition of the fabricated 
vessel. The inspection shall include a volumetric and surface inspection by the selected NDI 
techniques. 


The selected NDI techniques and inspection sensitivity for the metallic liner shall be according to 
Section 4.5.2 when safe-life demonstration is required. 


The NDI techniques selected for inspecting the composite overwrap of pressure vessels shall be 
according to Section 4.5.2. 


5.1.2 Proof Testing 

The COPY shall be proof tested to a minimum pressure of: 

P =(1 + Burst Factor)/2 X MEOP (for a burst factor less than 2.0) or 

=1.5 X MEOP for a burst factor equal to or greater than 2.0 

where MEOP is the Maximum Expected Operating Pressure. Unless otherwise stated, the 
duration of the proof test shall be sufficient to verify pressure stability. The COPV shall not leak, 
rupture, or experience detrimental deformation during proof testing. Proof-test fluids shall be 
compatible with the structural materials used in the COPV and not pose a hazard to test 
personnel. The proof test fixture shall emulate the structural response or reaction loads of the 
flight mounting where COPV mounting induces axial or radial restrictions on the pressure driven 
expansion of the vessel. The temperature shall be consistent with the critical use temperature, or 


test pressures shall be suitably adjusted to account for worst-case temperature effects on static 
strength and/or fracture toughness 


5.1.3 Leak Testing 
The COPY shall be leak tested at MEOP or greater. 
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5.2 Qualification Testing 


Qualification tests shall be conducted to demonstrate that all design requirements are met. The 
qualification test procedure shall be approved by the procurement agency prior to the start of 
qualification testing. 


As a minimum, the following tests shall be conducted on all new or substantially modified 
COPV designs: 


Safe-life demonstration per Section 5.2.1, or LBB [leak-before-burst] demonstration according to 
Section 5.2.2. 


(a) Acceptance test per Section 5.1 

(b) Pressure cycle testing per Section 5.2.3 

(c) Vibration/External load testing according to Section 5.2.4 
(d) Leak testing according to Section 5.1.3 

(e) Burst testing according to Section 5.2.5 


Qualification testing of COPVs that are similar to previously qualified vessels may be reduced 
subject to the approval of the procurement agency and appropriate range safety authority. 


If required, damage tolerance testing shall be conducted according to Section 4.2.10.3. The test 
article(s) may be the same as the ones used previously or may be separate as defined in the test 
plan. 


When conducting qualification testing, the test fixtures support structures, and methods of 
environmental application shall not induce erroneous or unrealistic test conditions for the 
intended application. The types of instrumentation for measuring stresses and displacements and 
their locations in qualification tests shall be based on the results of the stress analysis (Section 
4.2.5). Additional instrumentation shall be installed to provide complete monitoring and control 
of the test fixtures and hardware including temperature, pressure, and other critical parameters. 
The instrumentation and test plan shall be formulated to provide sufficient data to ensure proper 
application of input loads, pressures, environments, and vessel responses to allow assessment 
against accept/reject criteria, which shall be established prior to test. The sequences, 
combinations, levels, and duration of loads, pressure, and environments shall demonstrate that 
design requirements have been met. 


5.2.5 Burst Test 


Burst testing shall be conducted to verify compliance to the burst factor requirement defined in 
Section 4.2.2 in compliance with the verification requirements of Table 1. 


The design burst should be maintained for a period of time sufficient to assure that the proper 
pressure is achieved. The vessel shall not burst prior to the end of this period of time. After 
demonstrating the burst pressure, the pressure shall be increased at a controlled rate until vessel 
burst occurs. 


The burst test fixture shall simulate the structural response or reaction loads of the flight 
mounting where COPV mounting induces axial or radial restrictions on the pressure driven 
expansion of the vessel. 
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ATTACHMENT #3 


JSC 65828 Rev. A — Structural Design Requirements and Factors on Safety 
for Spaceflight Hardware 


3.3. PURPOSE 


This document establishes the structural requirements for human-rated spaceflight hardware 
including launch vehicles, spacecraft and payloads. These requirements are applicable to 
Government Furnished Equipment activities as well as all related contractor, subcontractor and 
commercial efforts. These requirements are not imposed on systems other than human-rated 
spacecraft, such as ground test articles, but may be tailored for use in specific cases where it is 
prudent to do so such as for personnel safety or when assets are at risk. 


The requirements in this document are focused on design rather than verification. 
Implementation of the requirements is expected to be described in a Structural Verification Plan 
(SVP), which should describe the verification of each structural item for the applicable 
requirements. The SVP may also document unique verifications that meet or exceed these 
requirements with NASA Technical Authority approval. 


3.2.8.5 Composite Overwrapped Pressure Vessels 


[STR0023] Composite Overwrapped Pressure Vessels (COPVs) shall comply with ANSI/AIAA-S- 
081, Standard for Space Systems — Composite Overwrapped Pressure Vessels (COPVs), with the 
following exceptions: 


a. Applicable loads and environments to be used in place of those derived following the AIAA 
standard are defined in JSC 65829, Loads and Structural Dynamics Requirements for Spaceflight 
Hardware. 


b. Applicable minimum factors of safety to be used in place of those defined in the AIAA standard 
are defined in Section 3.3.1.6. 


c. Applicable design pressures to be used in place of those defined in the AJAA standard are defined 
in Section 3.2.8.2. 


Rationale: The AIAA standard for COPVs is tailored to account for human spaceflight safety factor 
requirements and design loads. 


3.3.1.6 Pressurized Hardware 


[STR0044] Pressurized hardware shall be designed and tested to the minimum factors of safety 
specified in Table 3.3.1.6-1. 


Rationale: Proof tests are conducted as an acceptance test of each production unit including the 
qualification article. The proof factor is the minimum required. A higher proof test factor may be 
determined by fracture mechanics analysis when the proof test is used for flaw screening. 


Additional factor of safety requirements for liquid propulsion engine structures and solid rocket 
motors are provided in Sections 3.3.1.7 and 3.3.1.8, respectively. 

MDP = Maximum Design Pressure 

MEP = Maximum External Pressure 

[STROO45] 
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Table 3.3.1.6-1 — Minimum Factors of Safety for Pressurized Hardware 


Composite Overwrapped Pressure Vessels (COPV) 


Proof Pressure 1.25 x MDP 
Design Burst Pressure 2.00 x MDP 
Negative Pressure Differential* 1.00 x MEP 


*Must be capable of withstanding maximum external pressure multiplied by ultimate factor of 
safety (Negative Pressure Differential) without collapse or rupture when internally pressurized to 
the minimum anticipated operating pressure. 
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Appendix D: 2012 White Paper on the Need for Advanced 
Engineering Methods 


Development of Computational Materials-Based Methods for the 
Design of Components Having Small Length Scale Features 


E.H. Glaessgen 

Assistant Branch Head 

Durability, Damage Tolerance and Reliability Branch 
NASA Langley Research Center 


R.S. Piascik 
NASA Technical Fellow for Materials 
NASA Engineering and Safety Center 


Materials Discipline Technology Gap: 


Standard engineering fracture and fatigue-based design methods are not applicable for an ever- 
increasing number of advanced design concepts. 


Background: 


Many of the advanced design concepts are becoming increasingly aggressive; requiring 
thinner/smaller features that are subjected to more demanding operating environments. 
Simultaneously, the requirements for their durability and reliability have increased. 


The issue is that NASA is now designing and building components that contain various features 
that are so small that local material behavior cannot be accurately predicted using standard 
methods. Typically, these component features have length scales that are of similar dimensions 
to those of the material’s microstructural features. Examples of newly designed critical 
components where standard methods are no longer applicable include: 


1. Radioisotope thermoelectric generators (RTGs) where the life-limiting component 
incorporates an extremely thin wall (300 microns or 0.012 inch); here, the thin wall was 
needed to meet heat transfer requirements. 


2. High pressure overwrapped pressure vessels (COPVs) where new concepts require liner 
wall thickness of 250 microns (0.010 inch); here, the thin liner wall is needed to meet 
weight requirements and is the life-limiting component. 


3. Microelectromechanical systems (MEMS) and micromachines where sub-components 
having dimensions of only a few microns may fail due to mechanical and/or thermal 
stresses; here, nanomechanics may be required to capture material response. 


4. Thermal barrier coatings where coatings are often only a few hundred microns (~0.010 
inch) thick; here, delamination and spallation are life-limiting failure modes. 


The absence of a life prediction method to address examples | and 2 has recently introduced a 
true risk where engineers have used inappropriate methods (linear elastic fracture mechanics, 
LEFM) to determine design life. LEFM is not an appropriate method for these length scales; 
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LEFM is based on continuum theory and breaks-down at small length scales where 
microstructural features become important. The use of LEFM will result in inaccurate 
predictions that may be unconservative and may unknowingly affect safety and mission success. 
Hence, it is becoming increasingly apparent that there is a growing technology gap where the 
material modeling capabilities of standard engineering practice are inadequate for a growing 
number of new designs. Currently, the only approach to offset these deficiencies is to conduct 
expensive and time-consuming component and full-scale testing of flight hardware. However, 
this philosophy is counter to the Agency’s direction to limit costly testing. 


Materials Technical Challenge - Develop Computational Materials-Based Methods for the 
Design of Components with Service Lives that cannot be Predicted by Standard Methods: 


The proposed effort will develop needed physics-based computational and experimental 
micromechanics methods that will become the basis for a new methodology that is required for 
design of components having extremely small features. Although these micromechanics 
methods have traditionally been limited to the research community, recent developments have 
enabled several of the methods to attain a level of maturity where they are becoming viable for 
application to address NASA requirements.! 


Unlike Traditional engineering methods that consider a material to be a continuous and 
homogenous body (a continuum), micromechanics methods interrogate materials response at a 
much more fundamental level. As a result, deformation and fracture can be investigated at the 
scale where the microstructrure (individual grains or crystals in a metal or fibers in a composite) 
are considered. Hence, micromechanics methods are more appropriate for predicting material 
failure at small length scales than their continuum analogues and are the basis for powerful 
predictive tools. Additionally, novel experimental methods are being developed to measure 
material responses at these small length scales and validate the computational models. When 
successfully applied to the RTG and COPV components mentioned previously, the 
micromechanics-based approach will enable development of the rigorous design tool(s) required 
for these special small-length-scale design applications. 


Far Reaching Impact of Computational Materials-Based Predictive Methods: 


The proposed development of physics-based micromechanics life prediction is based on a 
growing National thrust toward a broad materials sub-discipline termed Integrated 
Computational Materials Engineering, ICME or computational materials.2> The computational 
materials vision is far reaching; computational materials-based methods will be the foundation 
for responding to future Agency needs from the materials discipline. Computational materials 
methods will lead to revolutionary rapid materials design and development, desktop materials 
properties characterization, physics-based life prediction methods, etc. A computational 
materials vision for the Agency will become a critical materials discipline tool for accelerating 
the introduction of new materials and for the physics-based prediction of material behavior that 
will lead to an unprecedented design capability for highly reliable deep space systems. 


Schedule, Budget and Other Resources: 


The proposed effort will develop micromechanics-based methods for use in certification of thin 
(t<0.5 mm) metallic structures such as RTG and thin COPY liners within the next 5 years. These 
methods for prediction and characterization of microstructurally small crack initiation and 
growth will extend recent work by members of the present team.*> 
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The major components of the work plan include: 


Lis Identification of a suitable focus problem (e.g., RTG or thin COPV liner) 


2s Modification of current micromechanics-based computational and experimental methods 
to address component-specific requirements 

3. Analysis and characterization of the component of interest; validation of analysis 

4, Generalization of the methods for a range of similar components 


LaRC will develop computational approaches in conjunction with a combined LaRC, GRC 
[Glenn Research Center], MSFC [Marshall Space Flight Center] and JPL [Jet Propulsion Lab] 
experimental effort to develop supporting experimental methods. A budget of 8 FTE [Full Time 
Equivalents] and $850K per year divided among the three participating centers will be required 
to accomplish this effort. Approximately 15-20 million c.p.u. hours per year on NAS/Pleiades 
supercomputer will also be required to support the simulations. 
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In some sense, there is confusion related to the 


. Fi ‘a Py Reality of interest 
words used and likely semantics issues that arise 


when a verification and validation (V&V) process Abstraction 

is advocated. The V&V process increases 
confidence in the model’s predictive capability 

through quantifying uncertainties and errors by “tedaing taoseing 


comparing results from the computational model to f C=) ( Persia ) 
other analytical solutions and to experimental Cole Implementation omnouale 
measurements. The ASME standard processes for ~* 


Be ab seas 3 2 < ~. Computational Pretminary Pristvarse 
V&V activities and products is shown in figure on ' C=) =-CE) “e 


the r ight. e atxlaton Caloulation Gapedanentation Experiment 


Here, some of the V&V terminology will be Ca) 
restated relative to developing a predictive Quorteaten 7 Ss Quicaton 
capability for progressive damage analysis of = p= 
composite structures. Within the V&V framework, how oS 

one of the challenges is defining appropriate tests —— HE te * 


(or experiments) to validate an analysis model. A 
model consists of the conceptual, mathematical, 
and computational (numerical) representations of 
the physical phenomena needed to represent SSS 
specific real-world conditions and scenarios. Thus, 

the model includes the geometrical representation of the domain, governing equations, boundary 
and initial conditions, loadings, constitutive models and related material parameters, spatial and 
temporal approximations (discretization), and numerical solution algorithms (implicit, explicit). 
This model should imitate or simulate the characteristics of the phenomena using a 
computational model. The computational model, usually in the form of numerical 
discretization, selected solution algorithm, and defined convergence criteria, is the numerical 
implementation of the mathematical model. The mathematical model involves the 
mathematical equations, boundary conditions, initial conditions, and modeling data needed to 
describe the conceptual model. The conceptual model is the collection of assumptions and 
descriptions of physical processes representing the solid mechanics behavior of the reality of 
interest (i.e., the physical system and its environment) from which the mathematical model and 
validation tests or experiments can be constructed using the computational model. Model 
validation is the process of determining the degree to which a model is an accurate 
representation of the real world from the perspective of the intended uses of the model (i.e., the 
experiment or test). This degree is assessed by one or more validation metrics using a 
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mathematical measure that quantifies the level of agreement between simulation outcomes and 
experimental/test outcomes. 


Validation tests generate data for the purpose of model validation and are defined and 
developed through cooperative and collaborative efforts between the test engineer and the 
analyst — usually they are different individuals. A series of such validation tests may be 
performed as the design progresses through a building-block development process (i.e., replica 
tests, and/or specific feature tests). Through the V&V, a model can be declared “validated for its 
intended use” once it has met the validation requirements. These requirements are established 
in the V&V plan and often include descriptions of specific tests that need to be run or other 
referents (i.e., data, theory or information) that need to be gathered, what specific measurements 
need to be acquired from the tests, as well as accuracy statements about the agreement between 
simulation outcomes and experimental/test outcomes. 


In my mind, each of the validation tests are guided by pre-test calculations. Pre-test 
calculations are those calculations or simulations performed using a computational model to 
help design a validation test and may include guidance in how the test should be run, what data 
should be collected, boundary conditions, pre-test configuration dimensions, generalized 
imperfections, load introduction, instrumentation layout, etc. In addition to pre-test calculations, 
pre-test measurements of the test articles and testing environment are also needed as an integral 
part of the process. Dimensional checks and variability, surface conditions (flat, smooth, 
imperfect), load introduction process, boundary restraints, and so forth aid in making the test and 
analysis answer the same question(s). 


A concern may [arise] when only one or two tests are 
performed resulting in the lack of a statistically relevant 
sample size due to cost or schedule or limited material or a 
combination of things resulting in an incomplete 
characterization of all uncertainties in the model or the test. 
Answers to questions like “what do I change to match the 
test?” or “which test do I match?” or “what physics was 
missed?” or “is the response output sensitive to input 
parameter variations?” or “is the model predictive of the next 
test?” The goal may then degenerate to putting “dots on the 
curve” based on a deterministic approach of analyzing each IaPuE Peretmetet 
individual test article and comparing the test response of that test article to the analysis result for 
that test article. Such a pre-test prediction approach only demonstrates that features of interest 
extracted from the simulation can be matched or correlated to test (i.e., putting “dots on the 
curve’). However, a predictive capability is not demonstrated, in my opinion, because the 
mechanics of the response may be misinterpreted or overshadowing by sensitivities in the model, 
variability in the test article, and uncertainties in the test data. That is, the uncertainties 
associated with this phenomenon and its modeling are either not well understood, not well 
represented, or both. A blind prediction approach is needed. 


Updated Model ~q,, 
2 


* 


Output Response 


Blind predictions (analyses accounting for uncertainties) and pre-test prediction (test article 
specific analysis) are those predictions from the computational model that are performed and 
reported without knowledge of or access to the actual test data (1.e., data from strain gages, 
displacement transducers, digital image correlation, etc.). Within the V&V framework and 
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documents, a prediction is the use of a model to foretell the state of a physical system under 
conditions for which the model has not been validated. The output from a model that calculates 
the response of a physical system before experimental data are available to the user. The analyst 
and the test engineer should be actively engaged in collaborative planning of both the analysis 
and test to ensure that both efforts are answering the same question and calculating or gathering 
the same data of interest. In some sense, the test and the analysis are both blind and pre-test 
calculations may help answer some questions for both testing and analysis. Experienced test 
engineers and seasoned analysts play crucial roles in this process. A validation test is more 
concerned that both the analysis and the test answer the same question rather than what the 
question is. The analysis team should be permitted to witness a test as long as no access to the 
test data is permitted. An analyst always learns from observing the tests that they are trying to 
simulate or mimic with an analysis model. Often it was an “I forgot...” that needed an answer. 
At other times, it helps understand the physical problem and the potential outcome or 
consequence of the how the analysis may be used. 


A prerequisite for access to the test data is the delivery of the blind predictions and/or a pre-test 
prediction. Usually these simulations are done prior to performing the test in an attempt to 
mimic the test. Again, a blind prediction that accounts for uncertainties is distinguished from a 
pre-test prediction that that attempts to mimic a particular test of a specific test article. 
Scheduling the completion of blind predictions or pre-test prediction should proceed the test but 
often times this sequencing may not be possible without impacting the testing schedule or the 
availability of key personnel. Test and analysis schedules should be independent. Some 
organizations or projects may mandate that testing does not begin until the blind predictions or 
pre-test prediction are delivered. However, the analysis team could be firewalled off from 
having access to the actual test data, discussions of observations from testing that would 
influence the modeling, and any post-test examination results (e.g., X-ray images, photographs, 
etc.) until the analysis results are delivered. Most likely, some aspects or phases of the testing 
will be simulated accurately, while other aspects may not due to missing physics, unknown 
unknowns, or just scatter in the data. 


Again, a blind prediction from a V&V perspective is more than a single 
response curve and more than just a single analysis of the specific test 
article (i.e., more than a pre-test prediction). Typically, various 
tolerances, probably not their statistical distributions, are known for the 
test article configuration. For example, dimensions + tolerance, 
thickness + tolerance, material properties (vendor values vs. independent 
verification), stiffener spacing and orientation on the panel, stacking 
sequence variations, thickness variations, generalized imperfections, etc. 
Blind predictions can be used to identify those features as modeling and 
analysis parameters (random variables) that may influence the structural 
response and assess response sensitivities. The number of suspected 
random variables (known unknowns) may be reduced based on the blind predictions 

(i.e., selection/evaluation/elimination of random variables) or even the verification process. A 
family of response curves (an ensemble of models) are then generated for example using the 
bounds of the random variables (modeling parameters) as well as using their nominal values. 
This process also requires the definition of metrics to be used to assess predictive capability. 
These metrics may include a physical response (e.g., pre-buckling stiffness, buckling load, peak 


Response 


Solutions for different 
combinations of uncertainty 
parameters 


Parameter 
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load, load at damage initiation in a particular mode, final failure load, etc.) or a statistical 
measure (e.g., cumulative distribution function). Understanding what those metrics are and what 
they represent is a key decision point that needs to be agreed upon jointly by the test engineer 
and the analyst and likely communicated to the decision makers. For example, does “failure 
load” mean first material failure, first load drop, loss of numerical convergence, or maybe some 
time limit? In addition, criteria for success need to be defined either in terms of the response 
metrics or a percentage of a single metric. The definition of these metrics and success criteria is 
crucial in evaluating the model’s performance and predictive capability as well as the validity of 
the test data. 


As aresult, the blind predictions generate a family of response Rony 

Se 
curves (called a “cone” here, others refer to them a “cloud” of “ous” Coy 
results) that ideally bound all test data — this is what would be 
called a “calibrated” probabilistic model. This calibrated 
probabilistic model includes a combination of modeling and 
analysis parameters (random variables), their actual or assumed 
Statistical distributions, and the ranges of those parameters. 
The bounds of the probabilistic model then encompass the 
bounds of the test data. The analyst should be able to identify 
within this probabilistic model a single set of random variable 
values such that the associated analysis result agrees with the 
mean of the test data — this has been called a “reconciled” model. The goal would be for the 
mean of the test results, for a given response metric(s), to agree with the analysis result using the 
reconciled model, which is a single instance of the calibrated probabilistic model. Then, a 
predictive capability has been demonstrated using a probabilistic model for a specific intended 
use of the model and given set of assumptions. In reality, the effort depends to some extent on 
the project schedule, budget, personnel, and experience as we attempt to quantify uncertainties 
and understand confidence levels that need to be communicated to the decision makers for risk 
assessment. 


Response 


Reconciled Model 
Mean of Test Data 


Mean of Calibrated Model 


Parameter (per discussions with Kenny Elliott) 


If only one test has been performed, establishing, or claiming to have established, a predictive 
capability is not realistic, but a blind prediction process (i.e., developing a calibrated 
probabilistic model) would still be useful. If the test result falls outside the “cone,” then 
obviously the model missed something and a post-test effort is needed. The process of adjusting 
numerical or physical modeling parameters in the computational model for the purpose of 
improving agreement with experimental data is often called model update or calibration. Often 
times, unknown unknowns emerge and cause problems (e.g., premature failure during a test, lack 
of convergence in analysis); however, they may also identify new aspects of the response that 
had not been anticipated or may not have been observed previously. Consequently, the analysis 
and test response should correlate with each other for some portions of the response domain 
(e.g., initial stiffness, knee in response curve), but may deviate in other regions due to changes in 
response, damage initiation and progression, changes in damage modes, etc. and thereby require 
re-visiting the model, the test, the random variables, and/or the metrics to determine the 
adequacy of the agreement and what it does and does not imply. 
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Update (Calibrate) Computational Model (if Needed) 


Computational System Response 
mode Quantity of Interest 


Result is a probabilistic model with a set of calibrated 
parameters and bounds (calibrated models) that 
includes one set as the reconciled model 


Validation 
Metric 


This computational model has achieved the specified level 
of accuracy in the solution of specified problems 
within the validation database (reproducible evidence) 


Validation 
Metric Result 


Pre-Experiment 
Calculations Needed 
Experiment 


for Design of the 


Post-Experiment Data 
Needed for the 
Computational Model 


Apply computational model 


Yes to application of interest within 
the intended use of the model 


Physical System Response 


Experiment 


Improve/Add Experimental Measurements (if Needed) 


Qberkampt, W.L. and M. F. Barone (2006), "Measures of Agreement Betw 
Computation and Experiment: Validation Metres." Journal of Computational Physics, 
Vol 217, No. 1, pp. 536. 


Often times verification steps have been performed previously and the analysis team needs only 
to adopt the previous modeling strategy for their application or validate a surrogate modeling 
approach as needed to make the analysis computationally tractable. For example, if the test 
article has an open hole, then a verification process should have addressed various element issues 
related to meshing, orientations, recovery of elastic response, sensitivity to element distortion, 
2D vs. 3D, single element through thickness vs. fiber-orientated overlaid mesh through the 
thickness, and so on, as well as issues related to damage initiation and propagation, numerical 
stability, and convergence. 


Another example is related to a stiffened panel test article. Presumably, the response before 
buckling is elastic with no damage. Are there modeling-related decisions that influence 
predicting elastic buckling (the eigen pairs)? Does the elastic post-buckling response show any 
sensitivity to modeling or analysis parameters such as type of imperfections, sign of the 
imperfections, solver parameters, or solver (implicit vs. explicit)? The analyst would want to 
understand any modeling/analysis sensitivities affecting an elastic prediction before doing a 
damage prediction. V&V to meet elastic response metrics provide a validated model for that 
intended use. Application of those same computational models to simulate damage initiation and 
progression would constitute a prediction (i.e., use of a validated model beyond its intended use), 
which may necessitate separate model validation steps. 


In summary, blind predictions from a V&V perspective add value and insight to the analysis 
effort. Test engineers and analysts need to work together and observe each other’s process, ask 
questions, and share thoughts in the development of validation tests through pre-test calculations. 
Access to test data by the analysis team is restricted until the blind predictions or pre-test 
prediction are completed and delivered. In my opinion, blind predictions are needed at each level 
of the building block so that various modeling strategies and model fidelity can be assessed and 
communicated to the next level. Each level of a testing building block has associated with it an 
independent modeling and analysis building block leading to credible predictions within each 
level. If model or tool cannot predict the response of a plate with a hole, what evidence will be 
needed to show that the response from a more complex structure analysis are credible? Each test 
has the potential to expose new unknown unknowns. Each analysis has the potential to expose 
modeling shortcomings or computational limits. Both exhibit uncertainties and limitations. 
Perhaps the more difficult task is establishing a working dialog and a realization that Murphy’s 
Law is still valid. 
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